This chapter covers VPC Flow Logs, a critical feature for security analysis and network troubleshooting in AWS. For the SAA-C03 exam, understanding flow logs is essential for designing secure and auditable architectures, as they enable monitoring of IP traffic to and from network interfaces in a VPC. Approximately 10-15% of exam questions touch on logging, monitoring, or security analysis, with flow logs being a core component. You will learn how to configure, interpret, and analyze flow logs to detect anomalies, troubleshoot connectivity, and meet compliance requirements.
Jump to a section
Think of VPC Flow Logs as a facility-wide security camera system for a large office building. Each network interface (ENI) is like a door or hallway intersection monitored by a camera. The camera records every person (packet) who passes through, noting the time, direction (in or out), and whether they were allowed (ACCEPT) or denied (REJECT). The recordings are stored in a central video archive (CloudWatch Logs or S3). Security personnel (analysts) can later review footage to investigate incidents, detect patterns (e.g., repeated failed entry attempts), or verify compliance. The cameras do not affect the movement of people—they merely observe and log. Similarly, VPC Flow Logs capture metadata about IP traffic without impacting network performance. Just as cameras have a fixed frame rate (capture every event), flow logs aggregate packets into flows based on a 5-tuple and log them after a timeout (default 600 seconds for active flows). If a door is rarely used, the camera records a brief clip (short flow). If a busy hallway, it records longer clips (long flows). Logs are written to CloudWatch Logs or S3, where you can search and analyze them using tools like Athena or third-party SIEM. The analogy holds: you cannot stop a person based on camera footage alone; you need a separate access control system (security groups/NACLs) to block. Flow logs are purely observational.
What VPC Flow Logs Are and Why They Exist
VPC Flow Logs capture metadata about IP traffic flowing through network interfaces in your VPC. They are not a packet capture tool; they do not record the payload of packets. Instead, they log information such as source and destination IP addresses, ports, protocol, packet and byte counts, and whether the traffic was accepted or rejected by security groups and network ACLs. Flow logs are essential for:
Security analysis: detecting unauthorized access attempts, data exfiltration, or compromised instances.
Network troubleshooting: identifying connectivity issues, asymmetric routing, or misconfigured security groups.
Compliance: maintaining audit trails of network traffic for regulatory requirements.
Flow logs can be created at three levels:
VPC level: captures all traffic for all ENIs in the VPC.
Subnet level: captures traffic for all ENIs in that subnet.
Network interface level: captures traffic for a specific ENI (e.g., an EC2 instance's primary ENI).
How It Works Internally
When an IP packet enters or leaves a VPC, the AWS hypervisor (Nitro or Xen) examines the packet and determines whether it is allowed or denied based on security group and NACL rules. The hypervisor then generates a flow log record for that packet's flow. A flow is defined by a 5-tuple: source IP, destination IP, source port, destination port, and protocol. All packets matching the same 5-tuple in the same direction within a certain time window are aggregated into a single flow log record.
Flow log records are written to Amazon CloudWatch Logs (default) or Amazon S3. The destination is configured when you create the flow log. You can also send flow logs to Amazon Kinesis Data Firehose for near-real-time analysis.
Key Components, Values, Defaults, and Timers
Aggregation interval: Flow logs capture metadata in windows of 10 minutes (maximum). However, records are published to CloudWatch Logs every 60 seconds (or 5 minutes for S3). The aggregation interval is the maximum time a flow can be active before it is logged. If a flow is idle for more than the default timeout (600 seconds for TCP, 60 seconds for UDP), it is considered terminated and a final record is logged.
Default flow timeout:
TCP: 600 seconds (10 minutes) unless a FIN or RST is seen.
UDP: 60 seconds (1 minute) because UDP is connectionless.
ICMP: 30 seconds.
Log format: Each record contains version (2 or 3), account ID, ENI ID, source IP, destination IP, source port, destination port, protocol, packets, bytes, start time, end time, action (ACCEPT/REJECT), log status (OK/NODATA/SKIPDATA), and more. Version 3 adds VPC ID, subnet ID, instance ID, and TCP flags.
Log status:
OK: data is being logged normally.
NODATA: no traffic for the ENI during the aggregation interval.
SKIPDATA: some records were skipped due to throttling or capacity limits (rare).
Pricing: Flow logs are charged per log record published to CloudWatch Logs (ingestion) and per GB of storage. There is no charge for the flow log service itself. S3 storage costs apply if using S3.
Limitations:
Flow logs cannot capture traffic to or from the following: Amazon DNS (e.g., 169.254.169.253), Amazon Windows license activation, Amazon Time Sync Service (169.254.169.123), DHCP traffic, traffic to the VPC router (implicit router), and traffic to the CloudWatch Logs endpoints (if using VPC endpoints).
Flow logs do not capture traffic between instances in the same VPC that are using private IPs, but they do capture traffic that goes through a VPC peering connection, transit gateway, or VPN.
Flow logs are not real-time; there is a latency of up to 10 minutes for CloudWatch Logs and up to 15 minutes for S3 (though typically faster).
Configuration and Verification Commands
To create a flow log using the AWS CLI:
aws ec2 create-flow-logs \
--resource-type VPC \
--resource-ids vpc-12345678 \
--traffic-type ALL \
--log-destination-type cloud-watch-logs \
--log-group-name my-flow-logs \
--deliver-logs-permission-arn arn:aws:iam::123456789012:role/FlowLogsRole--traffic-type can be ACCEPT, REJECT, or ALL.
--log-destination-type can be cloud-watch-logs or s3.
For S3, use --log-destination with the bucket ARN: arn:aws:s3:::my-bucket/prefix/.
To verify flow logs are active:
aws ec2 describe-flow-logs --filter "Name=resource-id,Values=vpc-12345678"To query flow logs in CloudWatch Logs Insights:
fields @timestamp, srcAddr, dstAddr, srcPort, dstPort, action, protocol
| filter action = "REJECT"
| sort @timestamp desc
| limit 100Interaction with Related Technologies
Security Groups and NACLs: Flow logs capture the action (ACCEPT/REJECT) as determined by the combination of security group and NACL rules. A packet must pass both to be accepted. If either denies, the action is REJECT. Flow logs help identify which rule is causing a block.
VPC Peering: Flow logs can capture traffic across a VPC peering connection if created on the VPC level or the ENIs involved.
Transit Gateway: Flow logs can be created on transit gateway attachments to monitor cross-VPC traffic.
AWS Network Firewall: Flow logs can be used in conjunction with Network Firewall to capture traffic that is inspected.
AWS Shield Advanced: Flow logs can help identify DDoS attack patterns.
Amazon Detective: Can ingest flow logs for security investigations.
Create Flow Log Destination
First, decide where to publish flow logs: CloudWatch Logs or Amazon S3. For CloudWatch Logs, create a log group in CloudWatch Logs (e.g., 'VPC-Flow-Logs'). For S3, create an S3 bucket with appropriate permissions. You must also create an IAM role that grants the VPC Flow Logs service permission to publish logs to the destination. The role must have a trust policy allowing 'vpc-flow-logs.amazonaws.com' to assume it. For S3, the bucket policy must allow 'logs.amazonaws.com' to write objects. This step is foundational; without correct permissions, flow logs will not deliver.
Create the Flow Log Resource
Use the AWS Management Console, CLI, or API to create a flow log. Specify the resource (VPC, subnet, or ENI), traffic type (ALL, ACCEPT, or REJECT), and the destination. For CloudWatch Logs, provide the log group name and IAM role ARN. For S3, provide the bucket ARN and an optional prefix. You can also specify the maximum aggregation interval (1 minute or 10 minutes). The CLI command 'create-flow-logs' returns a flow log ID (e.g., 'fl-12345678'). Once created, the flow log begins capturing traffic within minutes.
Monitor Flow Log Delivery
After creation, check the flow log status using 'describe-flow-logs'. The 'FlowLogStatus' field should be 'ACTIVE'. If it shows 'ERROR', check the IAM role permissions or S3 bucket policy. In CloudWatch Logs, you can view log streams (one per ENI) and search for log events. Use CloudWatch Logs Insights to query records. For S3, flow logs are delivered as gzipped text files (or Parquet format if enabled) in a folder structure based on the prefix and date. Files are typically delivered every 5 minutes.
Analyze Flow Log Data
Flow logs contain fields like 'version', 'account-id', 'interface-id', 'srcaddr', 'dstaddr', 'srcport', 'dstport', 'protocol', 'packets', 'bytes', 'start', 'end', 'action', 'log-status'. The 'action' field indicates whether the traffic was ACCEPTED or REJECTED. Use this to identify blocked traffic (REJECT) that may indicate security issues. For example, a high number of REJECT records to a specific IP may indicate a scan. You can also calculate total bytes transferred per destination for cost analysis. Tools like Amazon Athena can query S3-based flow logs using SQL.
Troubleshoot with Flow Logs
When troubleshooting connectivity, check if flow logs show ACCEPT or REJECT for the expected traffic. If REJECT, examine security group and NACL rules. If ACCEPT but no response, the issue may be at the application layer. Flow logs also help detect asymmetric routing: if a request is ACCEPTED but the return is REJECTED, check route tables and security group statefulness. Flow logs do not capture traffic to/from certain AWS services (e.g., DNS, DHCP), so missing logs for those are normal. For missing logs overall, verify the ENI is attached to a running instance and that the flow log is active.
In enterprise environments, VPC Flow Logs are a cornerstone of security operations. Consider a financial services company that must comply with PCI DSS. They enable flow logs on all VPCs and publish them to a centralized S3 bucket with server-side encryption. An AWS Lambda function is triggered on new log files to parse and load them into Amazon Elasticsearch Service (OpenSearch) for real-time dashboards. Security analysts monitor for anomalies like outbound traffic to known malicious IPs or large data transfers to unusual destinations. They also use Amazon Athena to run periodic queries to verify that no traffic is flowing to unauthorized regions. A common misconfiguration is forgetting to enable flow logs on new VPCs created via Infrastructure as Code (e.g., Terraform). To catch this, they use AWS Config rules to flag VPCs without flow logs.
Another scenario: a SaaS provider uses flow logs to troubleshoot customer-reported connectivity issues. When a customer says their application cannot reach an EC2 instance, the support engineer queries CloudWatch Logs Insights for the customer's source IP. If the logs show REJECT, they know a security group or NACL is blocking. If ACCEPT but no application response, they escalate to the application team. They also use flow logs to detect DDoS attacks: a sudden spike in REJECT traffic from many source IPs indicates a SYN flood. They then deploy AWS Shield Advanced and WAF to mitigate.
A third scenario: a large e-commerce platform uses flow logs for cost allocation. They tag each ENI with a cost center and use flow logs to calculate data transfer costs per service. They publish flow logs to S3 and use AWS Glue to catalog them, then query with Athena. This allows them to identify which microservices are generating the most inter-VPC traffic and optimize placement. A common mistake is not enabling flow logs on all ENIs, especially for NAT gateways and load balancers, missing important traffic. Also, they learned to set appropriate retention policies on CloudWatch Logs to avoid unexpected costs.
On the SAA-C03 exam, VPC Flow Logs appear in questions about security analysis, troubleshooting, and compliance. The specific objective code is Domain 1: Secure Architectures, Objective 1.1: 'Design secure access to AWS resources.' Flow logs are a key tool for auditing network traffic.
Common wrong answers and why candidates choose them: 1. 'Flow logs capture packet payloads' – candidates confuse flow logs with packet capture tools like VPC Traffic Mirroring. Flow logs only capture metadata, not payload. 2. 'Flow logs can be used to block traffic' – candidates think flow logs are reactive; they are purely observational. To block, you need security groups or NACLs. 3. 'Flow logs are real-time' – candidates assume logs appear instantly. In reality, there is up to 10 minutes delay for CloudWatch Logs and up to 15 minutes for S3. 4. 'Flow logs capture all traffic, including to AWS DNS' – the exam tests that traffic to 169.254.169.253 (DNS) is not logged. Similarly, DHCP and Windows activation traffic are excluded.
Specific numbers and terms that appear on the exam: - Default flow timeout for TCP: 600 seconds (10 minutes). - UDP timeout: 60 seconds. - ICMP timeout: 30 seconds. - Log format version: 3 (includes instance ID, VPC ID, subnet ID, TCP flags). - Traffic types: ACCEPT, REJECT, ALL. - Destinations: CloudWatch Logs, S3, Kinesis Data Firehose. - Resources: VPC, subnet, network interface.
Edge cases and exceptions the exam loves to test: - Flow logs do not capture traffic to/from the following: Amazon DNS, Amazon Windows license activation, Amazon Time Sync Service, DHCP, VPC router, and CloudWatch Logs endpoints (if using VPC endpoints). - If you create a flow log on a VPC but also have a subnet-level flow log, both are logged separately (duplicate records). - Flow logs are not supported for all instance types (e.g., some older types may not support). - When using S3, you can enable Parquet format and Hive-compatible partitions for querying with Athena. - Flow logs can be published to a cross-account S3 bucket if the bucket policy allows.
How to eliminate wrong answers: - If the question mentions 'blocking traffic' or 'preventing attacks', the answer is not flow logs; it's security groups, NACLs, or WAF. - If the question asks for 'packet contents' or 'payload', eliminate flow logs. - If the question implies instant visibility, eliminate flow logs due to delay. - If the question involves 'troubleshooting DNS', flow logs won't help because DNS traffic is not logged.
VPC Flow Logs capture metadata (not payload) for IP traffic to/from ENIs in a VPC, subnet, or specific ENI.
Flow logs can be published to CloudWatch Logs, S3, or Kinesis Data Firehose.
Default TCP flow timeout is 600 seconds; UDP is 60 seconds; ICMP is 30 seconds.
Traffic to/from Amazon DNS (169.254.169.253), DHCP, and Time Sync Service is NOT logged.
Flow logs are not real-time; expect up to 10 minutes delay for CloudWatch Logs and up to 15 minutes for S3.
Use flow logs to identify REJECTED traffic for security analysis and ACCEPTED traffic for connectivity verification.
Flow logs cannot block traffic; use security groups or NACLs for access control.
You can query flow logs in CloudWatch Logs Insights or Amazon Athena (if stored in S3).
Flow logs support version 3 which includes instance ID, VPC ID, subnet ID, and TCP flags.
Flow logs are regional; they only capture traffic within the same region.
These come up on the exam all the time. Here's how to tell them apart.
VPC Flow Logs
Captures metadata only (no payload).
No impact on instance performance.
Free of charge (pay only for storage in CloudWatch/S3).
Default aggregation interval up to 10 minutes.
Ideal for security analysis and troubleshooting.
VPC Traffic Mirroring
Captures full packet payloads.
May impact instance performance due to mirroring overhead.
Charged per GB of mirrored traffic.
Near real-time (low latency).
Ideal for deep packet inspection and intrusion detection.
Mistake
VPC Flow Logs capture packet payloads.
Correct
Flow logs only capture metadata (source/destination IP, ports, protocol, packet/byte counts, action). They do NOT capture the payload of packets. For payload capture, use VPC Traffic Mirroring or network appliances.
Mistake
Flow logs can be used to block malicious traffic automatically.
Correct
Flow logs are passive logs; they cannot block traffic. To block traffic, you must configure security groups, network ACLs, AWS Network Firewall, or AWS WAF. Flow logs only record whether traffic was accepted or rejected.
Mistake
Flow logs are real-time and appear instantly.
Correct
Flow logs are aggregated and published with a delay. For CloudWatch Logs, records appear within 10 minutes (typically 1-2 minutes). For S3, files are delivered every 5 minutes (up to 15 minutes delay).
Mistake
Flow logs capture all traffic, including DNS and DHCP.
Correct
Flow logs do not capture traffic to/from Amazon DNS (169.254.169.253), Amazon Time Sync Service (169.254.169.123), DHCP, Windows license activation, or traffic to the VPC router. These are excluded by design.
Mistake
Flow logs on a VPC capture traffic for all ENIs, including those in peered VPCs.
Correct
A VPC-level flow log captures traffic for ENIs in that VPC only. To capture traffic across a VPC peering connection, you need flow logs on both VPCs (or on the peering connection itself, if supported).
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
VPC Flow Logs capture metadata (source/destination IP, ports, protocol, packets, bytes, action) and do not capture packet payloads. They are free (pay only for storage) and have no performance impact. VPC Traffic Mirroring captures full packet payloads for deep packet inspection, but incurs charges per GB of mirrored traffic and may impact instance performance. Flow logs are for security analysis and troubleshooting; Traffic Mirroring is for intrusion detection and compliance.
Flow logs are aggregated and published to CloudWatch Logs every 60 seconds (for 1-minute aggregation) or up to 10 minutes (for 10-minute aggregation). However, there is an additional latency of a few minutes. In practice, you can expect logs to appear within 1-10 minutes after traffic occurs. For S3, logs are delivered every 5 minutes.
Yes, flow logs capture traffic that goes through an Internet Gateway if the flow log is enabled on the VPC or the ENI. The logs will show the private IP of the instance as source/destination and the public IP of the remote host. Note that the action field indicates whether the traffic was allowed or denied by security groups and NACLs.
The 'log-status' field indicates the logging status: 'OK' means data is being logged normally; 'NODATA' means no traffic was recorded for the ENI during the aggregation interval; 'SKIPDATA' means some records were skipped due to internal capacity issues (rare). You can filter for 'NODATA' to identify idle ENIs.
Yes, you can enable flow logs on existing VPCs, subnets, or ENIs at any time. There is no downtime. Once enabled, flow logs begin capturing traffic within minutes. You can also disable or delete flow logs later.
Flow logs are supported for most current generation instance types. However, some older instance types (e.g., T1 micro, M1, C1) may not support flow logs. Additionally, flow logs require the ENI to be attached to a running instance. For unsupported types, the flow log will show 'NODATA' or no records.
First, create a table in Athena using the schema matching the flow log version. For version 2, you can use a CREATE TABLE statement with columns like 'version', 'account-id', 'interface-id', 'srcaddr', etc. Specify the S3 location and use the OpenCSV SerDe. Then run SQL queries, e.g., 'SELECT srcaddr, dstaddr, action FROM flow_logs WHERE action = 'REJECT'.' Enable Hive-compatible partitions for better performance.
You've just covered VPC Flow Logs for Security Analysis — now see how well it sticks with free SAA-C03 practice questions. Full explanations included, no account needed.
Done with this chapter?