Network logging, monitoring, and audit are the tools that give you a complete, historical record of everything happening inside your network. Without them, you would be flying blind, unable to troubleshoot problems, detect security breaches, or prove compliance. For the ANS-C01 exam, understanding these services is critical because they form the foundation of network visibility, security analysis, and operational troubleshooting.
Jump to a section
A simple way to picture Network Logging, Monitoring, and Audit
A private investigator's case file is a complete record of every activity related to a case. A good PI doesn't just solve the case; they meticulously document every phone call, every surveillance note, every witness interview, and every photograph. This detailed file is the 'logging and monitoring' of the investigation.
Network logging and monitoring works the same way. VPC Flow Logs are like the PI's daily surveillance log – they capture every single 'person' (network packet) that enters or leaves the 'building' (a subnet or network interface). They record who came, who left, who tried to get in but was turned away, and the exact time. CloudWatch is the PI's command centre – a central dashboard that collects all these logs, sets up alarms (like a silent alarm if someone tries to break in after hours), and stores everything for later review. Traffic Mirroring is like hiring a second PI to secretly follow a specific suspect; it copies all network traffic from a particular server or application and sends an identical copy to a security appliance for deep analysis, without interrupting the original communication.
Without these tools, investigating a network issue would be like a PI trying to solve a case with no notes, no timeline, and no record of who was seen where. You would have no evidence, no history, and no way to prove what happened. The case file – the logs and monitoring – is the only thing that provides the complete, irrefutable picture.
Think of your network as a busy city. Cars (data packets) are constantly moving between houses (servers) and businesses (applications). Network logging, monitoring, and audit are your traffic cameras, your traffic counters, and your police report system combined.
VPC Flow Logs are like a security camera at every street intersection and building entrance. They capture metadata about every 'car' that passes – its source address (where it came from), destination address (where it was going), the port (which door it used, like UDP 53 for DNS or TCP 443 for HTTPS), the protocol (TCP, UDP, ICMP), and whether the car was allowed through (ACCEPT) or rejected (REJECT). A 'flow' is a conversation between two endpoints. Flow logs capture this information for all network interfaces in a VPC (Virtual Private Cloud), which is your private section of the AWS cloud. You enable them per network interface, per subnet, or per VPC. They are published to CloudWatch Logs or Amazon S3 (Simple Storage Service) for storage and analysis.
CloudWatch is your central command centre. It is a monitoring and observability service that collects data from your AWS resources and applications. There are two main components relevant here: - CloudWatch Logs: This is where VPC Flow Logs are sent. It stores, monitors, and searches your log data. You can create 'metric filters' – like setting up a rule that counts every time a 'REJECT' log appears from a specific IP address. This turns raw log data into a numerical metric (a count) that can trigger an alarm (a CloudWatch Alarm). - CloudWatch Alarms: These watch a single metric over a specified time period and perform an action when the metric crosses a threshold. For example, if 100 REJECT packets appear in 5 minutes, the alarm can trigger an SNS (Simple Notification Service) notification to send you an email or a text message.
Traffic Mirroring is the most powerful tool. Imagine you want to analyse the exact behaviour of a specific suspect car without affecting its movement. Traffic Mirroring creates a clone of all packets sent and received by a particular Elastic Network Interface (ENI – a virtual network card attached to an EC2 instance). This clone is sent to a traffic monitoring appliance (like a security appliance from a third-party vendor or a network analysis tool) for deep packet inspection. It does not slow down the original traffic or change its path; it is a passive copy. This is incredibly useful for:
Intrusion detection and prevention
Performance monitoring of specific applications
Troubleshooting network performance issues
Compliance auditing
Why do these exist? Before these services, debugging a network issue in the cloud required manually sifting through server logs, capturing packets with tcpdump on individual instances (a very tedious and intrusive process), or setting up complex third-party monitoring solutions. AWS native services provide a scalable, managed, and integrated way to achieve full visibility without the operational overhead.
When do you use each? - VPC Flow Logs are for general traffic visibility, security auditing, and troubleshooting connectivity issues. - CloudWatch is for aggregating logs, creating metrics, setting alarms, and having a central dashboard. - Traffic Mirroring is for deep packet analysis, specific security threat detection, and advanced troubleshooting where you need the full packet payload, not just metadata.
Enable VPC Flow Logs on a Subnet
First, you decide which traffic you want to monitor. You enable VPC Flow Logs at the subnet level, which automatically captures all traffic on all network interfaces within that subnet. You specify a destination: either a CloudWatch Logs log group (for real-time analysis) or an S3 bucket (for long-term storage and batch analysis).
Create a CloudWatch Metric Filter
Once flow logs are streaming into CloudWatch Logs, you create a metric filter. For example, you create a filter that looks for the word 'REJECT' in the log entries. Every time a log entry contains 'REJECT', it increments a custom metric called 'RejectedPackets'. This turns unstructured log data into a structured number.
Create a CloudWatch Alarm on the Metric
With the 'RejectedPackets' metric now available, you create a CloudWatch Alarm. You set a threshold: if the metric exceeds 10 within a 5-minute period, the alarm goes to the 'ALARM' state. You then configure the alarm to send an SNS notification to your mobile phone when it triggers.
Set up Traffic Mirroring for a Specific Server
When you need deep analysis (e.g., to see a suspicious sequence of packets), you identify the EC2 instance's ENI (the source). You create a Traffic Mirror filter to capture only TCP traffic on port 443 (HTTPS). You specify a Target (a Network Load Balancer that forwards to a security appliance). You create the mirror session. Now all encrypted HTTPS traffic to that server is copied to the security tool.
Analyse Mirrored Traffic and Generate a Report
The security appliance receives the mirrored packets. It reassembles the TCP streams and performs deep packet inspection. It can detect a SQL injection attempt or a brute-force password guess. You examine the report, identify the attacker's IP, and update the security group rules to block that IP.
Imagine you are a network engineer at 'ShopGlobalNow', an e-commerce company. You are on call when you get an alert: 'Checkout latency is spiking.' Customers are abandoning their carts because the payment page is slow. You need to diagnose the problem quickly.
Step 1: Enable and Check VPC Flow Logs. You have already enabled VPC Flow Logs for the subnet containing your payment application servers. You query the flow logs in CloudWatch Logs Insights (the log query tool) for the last 15 minutes. You filter for the destination IP of the payment gateway and see a high number of packets. Crucially, you notice a pattern: all the REJECT packets are from a specific external IP address. This could be a DDoS (Distributed Denial of Service) attack, but you need more information.
Step 2: Set a CloudWatch Alarm. You don't have an alarm for this yet, but you can quickly create a metric filter that counts all ACCEPT and REJECT packets from the payment subnet. You then set a CloudWatch Alarm that triggers a Lambda function to automatically update the security group to block that suspicious IP address. This is an automated response to a network issue.
Step 3: Use Traffic Mirroring for Deep Analysis. You suspect the performance issue might be a bug in a new version of the payment application, not an attack. You enable Traffic Mirroring on the ENI of one of the payment servers and send the mirrored traffic to a network performance monitoring tool (like a packet analyser). Because Traffic Mirroring sends a full copy of packets, you can see inside each packet – you can see the full TCP handshake, the application-layer data (like the payment request payload), and the response times from the database server. You discover that the application is sending an unnecessarily large query to the database every time a user tries to pay, causing the database to be slow. You did not disturb the original traffic at all; you just watched a copy.
Step 4: Root Cause and Fix. With the data from Traffic Mirroring, you can definitively say the issue is in the application code, not the network. You notify the development team, who fix the query. The next day, latency is back to normal. You then create a permanent CloudWatch Dashboard that shows:
A line graph of packet ACCEPT/REJECT counts for the payment subnet.
A histogram of latency as measured by your mirrored traffic logs.
A log table showing top source IPs hitting the payment servers.
This real-world scenario shows how these tools work together: Flow Logs for visibility, CloudWatch for alerting and automation, and Traffic Mirroring for deep, non-intrusive diagnosis.
The ANS-C01 exam loves to test the differences between these three services and when to use each. You will almost certainly see at least 3-4 questions on this topic. The traps are subtle.
What they test: - VPC Flow Logs metadata: You must know what they capture (5-tuple – source IP, destination IP, source port, destination port, protocol; plus action ACCEPT/REJECT, packet number, byte number, and flow start/end time). They do NOT capture packet payloads. They do NOT capture traffic from AWS DNS servers, Amazon DHCP servers, or traffic to Amazon's metadata IP (169.254.169.254) by default (they do in 2024, but you need to know the default exclusion). - CloudWatch Logs and Logs Insights: You must know that Logs Insights is a query language to search and analyse flow logs. Know how to create metric filters from log data. Know that you can send flow logs to S3 for long-term storage and use Athena (SQL query service) to query them. - Traffic Mirroring specifics: You must know that it is a per-ENI feature, not per-subnet. It requires a mirror target (a Network Load Balancer or an ENI of another instance) and a mirror filter (which defines what traffic to mirror). You can mirror all traffic or filter by protocol, source, destination, etc. It does not support all instance types – only Nitro-based instances (the newer hypervisor). It also does not support traffic to the metadata endpoint.
Common traps: - Confusing VPC Flow Logs with Traffic Mirroring. A question will describe a scenario requiring deep packet inspection (like seeing the contents of a SQL query). The correct answer will involve Traffic Mirroring, not Flow Logs. Trap: Flow Logs can tell you something happened, but not the content. - Confusing CloudWatch Logs with CloudWatch Metrics. Flow logs go to CloudWatch Logs; metric filters turn log data into CloudWatch Metrics. An alarm watches a metric, not a log directly. - Thinking VPC Flow Logs capture all traffic. They do not capture traffic to 169.254.169.254 (metadata), 169.254.169.123 (Amazon Time Sync Service), or AWS DNS (the VPC resolver's IP). - Believing Traffic Mirroring is free. It is a feature that costs money per hour you run it. The exam might ask you to choose a cost-effective solution; Traffic Mirroring is only used when you need deep inspection.
Key definitions to memorise: - ENI (Elastic Network Interface): A virtual network card in the cloud. - Nitro-based instances: The new generation of EC2 instances that support Traffic Mirroring. - 5-tuple: Source IP, destination IP, source port, destination port, protocol. - Metric filter: A rule that extracts a pattern from log data and converts it into a numerical metric.
VPC Flow Logs capture network metadata (source IP, destination IP, ports, protocol, action) but never packet payloads.
CloudWatch Alarms watch metrics, not logs directly; you must create a metric filter to turn log data into a metric.
Traffic Mirroring copies full packets from an Elastic Network Interface to a monitoring appliance for deep packet inspection without affecting original traffic.
VPC Flow Logs are enabled per network interface, per subnet, or per VPC, and can be published to CloudWatch Logs or Amazon S3.
Traffic Mirroring requires a Nitro-based EC2 instance (not all instance types support it) and has an additional cost.
To analyse VPC Flow Logs with SQL queries, publish them to S3 and use Amazon Athena, not just CloudWatch Logs Insights.
These come up on the exam all the time. Here's how to tell them apart.
VPC Flow Logs
Captures only metadata (5-tuple, bytes, action).
Enabled per ENI, subnet, or VPC.
Cost is per GB of log data published.
Traffic Mirroring
Captures full packet payload for deep inspection.
Enabled per ENI only (not per subnet).
Cost is per hour of mirroring plus data processed.
CloudWatch Logs
Stores raw text data (log events).
Requires metric filters to extract numeric values.
Primarily for storage, search, and query.
CloudWatch Metrics
Stores numeric time-series data.
Used directly by alarms and dashboards.
Primarily for alerting and visualisation.
Flow Logs to CloudWatch Logs
Allows real-time querying with Logs Insights.
Best for proactive monitoring and alerting.
Can trigger metric filters and alarms instantly.
Flow Logs to S3
Best for long-term, cost-effective storage.
Queried using Amazon Athena (SQL) in batch mode.
Better for compliance and audit retention.
Mistake
VPC Flow Logs capture the full packet, including the application data (e.g., the contents of an HTTPS request).
Correct
VPC Flow Logs only capture metadata (the 5-tuple, packet count, byte count, and action). They never capture the packet payload. For payload analysis, you need Traffic Mirroring.
This misconception arises because people think 'logging' means 'recording everything'. In networking, metadata is often sufficient for security audits, and capturing payloads is more expensive and privacy-sensitive.
Mistake
You can use CloudWatch Alarms directly on VPC Flow Logs without creating a metric filter.
Correct
An alarm watches a CloudWatch Metric. VPC Flow Logs are stored in CloudWatch Logs. You must first create a metric filter to extract a numeric value from the logs (e.g., count of REJECTs) and that metric can then be used in an alarm.
This mistake happens because the exam tests the separation between logs and metrics. Beginners think 'I want an alarm on a log' without understanding the transformation step.
Mistake
Traffic Mirroring is a free feature included with all EC2 instances.
Correct
Traffic Mirroring incurs additional charges based on the number of mirrored hours and the amount of data mirrored. It is not free. Also, only Nitro-based instances support it.
The word 'mirroring' sounds like a simple toggle; people assume it is a built-in feature with no extra cost, like enabling flow logs (which also have a small cost).
Mistake
VPC Flow Logs capture all traffic in the VPC, including traffic to the internet gateway, NAT gateway, and VPN connections.
Correct
VPC Flow Logs capture traffic that traverses the network interfaces they are enabled on. They do not capture traffic to or from the internet gateway itself, or to NAT gateways (unless you enable flow logs on those specific resources). They do capture traffic that goes through them to egress.
Beginners think 'VPC Flow Logs' means 'all traffic in the VPC', but the logs are per-interface. Traffic that bypasses an interface (like traffic directly to the internet gateway's management plane) is not captured.
Mistake
CloudWatch Logs is only for VPC Flow Logs and cannot integrate with other AWS services.
Correct
CloudWatch Logs is a central logging service that can ingest logs from numerous AWS services (Lambda, CloudTrail, API Gateway, etc.) and even from on-premises servers via the CloudWatch Agent. It is a universal log sink.
This is a scope misunderstanding. New learners often think a service is single-purpose (Flow Logs -> CloudWatch), but CloudWatch is a versatile platform.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
Yes. VPC Flow Logs capture both ACCEPT and REJECT actions. If a packet is dropped by a security group or network ACL, the flow log will show the action as 'REJECT' for that specific flow, which is incredibly useful for troubleshooting connectivity issues.
No. VPC Flow Logs only capture metadata (the 5-tuple, packet count, byte count, and action). They do not capture the packet payload, so you cannot see application-layer data like URLs or HTTP headers. You need Traffic Mirroring for that level of detail.
CloudWatch Logs stores raw log data (like text from VPC Flow Logs). CloudWatch Metrics stores numeric time-series data (like 'RejectedPacketCount'). You cannot create an alarm directly on a log; you must first create a metric filter that extracts a numeric value from the log and publishes it as a metric.
No. Traffic Mirroring creates a passive copy of the packets and sends the copy to the mirror target. It does not intercept, modify, or delay the original traffic. The original traffic continues on its normal path without any impact.
No. When you create a VPC Flow Logs configuration, you choose a single destination: either a CloudWatch Logs log group or an S3 bucket. You can set up multiple flow logs on the same interface to send to different destinations, but that uses more resources and costs more.
The VPC Flow Log delivery will stop. The flow log will remain in an 'ACTIVE' state in the console, but no new log events will be delivered. You must either create a new log group and update the flow log configuration, or delete the flow log entirely.
You've finished Network Logging, Monitoring, and Audit. Continue through the ANS-C01 study guide to build a complete picture of the exam.
Done with this chapter?