Centralizing VPC Flow Logs to S3 Cross-Account

A company has a centralized logging account that receives VPC flow logs from all accounts. The logs are stored in an S3 bucket. The security team needs to analyze these logs to detect anomalous traffic patterns. Which solution provides the most cost-effective and scalable analysis?

⚠ Common exam trap

Watch out — candidates often choose Redshift or Kinesis because they associate 'analysis' with traditional data warehouses or real-time processing, overlooking that Athena's serverless, pay-per-query model is the most cost-effective and scalable for ad-hoc SQL analysis of data already in S3.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Use Amazon Athena to run SQL queries directly on the S3 bucket containing the flow logs.

Amazon Athena is the most cost-effective and scalable solution because it allows querying VPC flow logs directly in S3 using standard SQL without requiring data loading or infrastructure management. Athena's serverless, pay-per-query model eliminates idle costs and scales automatically to handle any volume of log data, making it ideal for ad-hoc security analysis of historical logs.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Use Amazon QuickSight to create dashboards from the flow logs.
Why it's wrong here
QuickSight renders visual dashboards from structured datasets; it cannot itself parse raw VPC flow log records at scale, so anomaly detection over billions of log lines needs Athena or OpenSearch first. It is tempting because QuickSight is the AWS BI tool for sharing interactive dashboards over aggregated data.
✓
Use Amazon Athena to run SQL queries directly on the S3 bucket containing the flow logs.
Why this is correct
Athena is serverless and queries data in place in Amazon S3, so there is no cluster to provision or data to load. Cost scales with bytes scanned, making it both scalable and cost-effective for analysing accumulated VPC flow logs.
✗
Set up Amazon Kinesis Data Analytics to process the flow logs in real time.
Why it's wrong here
Kinesis Data Analytics processes streaming data continuously, incurring always-on compute charges even when no anomalies occur, and flow logs are already delivered to S3 rather than a stream. It suits real-time transformation of live streaming data, not batch analysis of stored logs.
✗
Load the flow logs into Amazon Redshift and run SQL queries.
Why it's wrong here
Redshift requires provisioning clusters and loading flow logs into its columnar storage, adding transformation and ongoing compute cost for sporadic security queries. It suits structured, high-concurrency analytical workloads with predictable query patterns, not ad hoc scanning of raw S3 logs.

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

Go deeper

Centralizing VPC Flow Logs to S3 Cross-Account

Why each option matters

Option-by-option breakdown

AWS S3 Storage Class Comparison

Related to this question