VPC Flow Logs and Packet Mirroring

Let's start with the network itself. A VPC (Virtual Private Cloud) is your private, isolated section of the cloud where you run your virtual machines, databases, and other services. These resources communicate with each other and with the internet using network traffic, which is divided into small chunks called packets. Each packet has a header with metadata (source and destination IP addresses, ports, protocol) and a payload containing the actual data being sent.

VPC Flow Logs are a feature from cloud providers (like Google Cloud) that captures metadata about every packet that flows through your VPC's network interfaces. Think of it as a log file that records each conversation: when it started, who was talking, what protocol they used, how much data was transferred, and whether the packet was allowed or blocked by firewall rules called VPC firewall rules. Flow logs do not capture the actual content of the communication — they only capture the metadata. This is critical because it means they are privacy-preserving and relatively cheap to store.

Here is how VPC Flow Logs work:

- Every network interface (called a virtual NIC or network interface card) in your VPC can generate flow log entries. - The log entries are pushed to a logging service like Cloud Logging, where you can analyse them or export them to a storage bucket for long-term retention. - Each log entry includes fields like: source IP, destination IP, source port, destination port, protocol (TCP/UDP/ICMP), bytes transferred, packets transferred, and the action taken (ACCEPT if the packet was allowed, DROP if it was denied by a firewall rule). - You can aggregate these logs by subnet, network, or specific instances. - Flow logs are typically sampled (for example, one log entry per 10 packets) to reduce cost and volume. You can adjust the sampling rate. - Flow logs are extremely useful for network troubleshooting (e.g., 'why can't my web server connect to the database?'), security auditing (e.g., 'who tried to connect to our internal server?'), and cost allocation (e.g., 'which VPC is using the most bandwidth?'). Now, Packet Mirroring is a completely different beast. Packet Mirroring creates a copy of every single packet that flows through a specific network interface and sends that copy to a collector (usually a virtual machine or a network appliance) for deep inspection. Unlike flow logs, packet mirroring captures the full packet — both the header and the payload. This means you can see the actual content of the data, such as the text of a web request or the bytes of a file transfer. How Packet Mirroring works:

- You create a mirroring policy that specifies which network interface (the 'mirror source') to copy traffic from. - You also specify a 'mirror destination' — a network endpoint that will receive and process the mirrored traffic. - The cloud provider's network infrastructure automatically copies each packet at the hypervisor level (the software that runs the virtual machine) and forwards the copy to the destination. - The original packet continues on its path undisturbed — the mirroring is passive and does not affect performance. - The mirrored traffic is typically sent to a network security appliance (like an intrusion detection system or IDS), a packet capture tool (like Wireshark), or a load balancer for analysis. - Packet Mirroring is used for deep security analysis, forensic investigations, protocol debugging, and when you need to see the actual content to identify malware or data exfiltration attempts. The key differences between the two tools:

- Data captured: Flow logs capture metadata only; Packet Mirroring captures the full packet including payload. - Storage cost: Flow logs are cheap and can store months of data; Packet Mirroring generates massive volumes of data (every packet) and is expensive to store long-term. - Use case: Flow logs for routine monitoring, alerting, and troubleshooting; Packet Mirroring for in-depth security analysis during incidents or compliance audits. - Performance impact: Both are passive, but Packet Mirroring uses more network bandwidth on the collector side. - Sampling: Flow logs are typically sampled; Packet Mirroring is not — it is a full copy of every packet. On the PCNE exam, you will encounter scenarios where you must choose the right tool. A common question: 'You need to see the payload of a suspicious connection to determine if it contains malware. Which tool do you use?' Answer: Packet Mirroring, because flow logs do not capture payloads. Another typical trap: 'You need to log all dropped packets for compliance. Which tool?' Answer: VPC Flow Logs (specifically with the action field set to DROP) because they are cheaper and store metadata for months, whereas packet mirroring is too expensive for continuous compliance logging.

Imagine you are working as a cloud network engineer for a mid-sized e-commerce company that runs its entire website on Google Cloud. Your VPC contains web servers, application servers, and a database server. One day, the security team reports unusual outbound traffic from a database server to an unknown IP address. They suspect a data breach. Your job is to investigate. Step one: Use VPC Flow Logs to get a timeline. You enable flow logs on the database server's network interface and export them to Cloud Logging. You filter by the database server's IP and look for outbound traffic to unknown IPs. The logs show that for the past three hours, the database server has been sending small packets to an IP in another country every 30 seconds. The flow logs tell you the source port, destination port, and protocol (TCP), and that the packets were accepted by the firewall. This gives you a pattern. Step two: Deploy Packet Mirroring for deep inspection. Because you need to see the actual content of those packets (to confirm if it is exfiltrated customer data), flow logs are useless — they only show metadata. You create a packet mirroring rule that copies all traffic from the database server's network interface and sends it to a virtual machine running an intrusion detection system (IDS). The IDS captures the packets and analyses them. Step three: The IDS reveals the payload contains encrypted data that matches the format of your customer database. You now have evidence of a breach. You can see the exact data being sent because packet mirroring captured the full payload. You then block the outbound traffic using a firewall rule and isolate the database server. Step four: For ongoing compliance, you enable VPC Flow Logs on all critical servers. You set up an automated alert in Cloud Logging that triggers if any server sends more than 1GB of outbound data in an hour (a potential exfiltration indicator). You also periodically enable Packet Mirroring for short durations (e.g., one hour per week) on the most sensitive servers to randomly audit traffic content. What you learn from this scenario: Flow logs are your everyday visibility tool — cheap, always on, good for spotting patterns and anomalies. Packet Mirroring is your forensic scalpel — expensive, used only when you need to examine the exact content. When the exam asks about incident response, you pick packet mirroring for payload analysis. When it asks about continuous monitoring for compliance logs, you pick flow logs. Another practical tip: you can combine both tools. For example, use flow logs to identify suspicious IPs, then temporarily enable packet mirroring to capture traffic to those IPs for forensic purposes.