# NSG flow logs

> Source: Courseiva IT Certification Glossary — https://courseiva.com/glossary/nsg-flow-logs

## Quick definition

NSG flow logs capture information about network traffic that is allowed or denied by a Network Security Group in Microsoft Azure. They show details like source and destination IP addresses, ports, protocols, and whether the traffic was permitted or blocked. These logs help administrators understand network behavior and detect potential security issues. They are a key tool for network monitoring in cloud environments.

## Simple meaning

Imagine you have a security guard at the entrance of a large office building. This guard keeps a detailed notebook of everyone who comes in or out, their name, the time they arrived, which floor they went to, and whether they were allowed in or turned away. That notebook is like an NSG flow log. In the cloud, an Azure Network Security Group (NSG) acts as that security guard for your virtual network. It checks every packet of data trying to enter or leave your network resources, like virtual machines, and decides whether to allow or deny it based on rules you set up. NSG flow logs are the records of all those decisions. They tell you which traffic was allowed, which was blocked, where it came from (IP address), where it was going, and what port and protocol were used.

These logs are not enabled by default; you have to turn them on for each NSG you want to monitor. Once enabled, the logs are sent to an Azure storage account, where you can download and analyze them, or send them to tools like Azure Monitor or a SIEM (Security Information and Event Management) system. This helps you see patterns, such as a sudden spike in blocked traffic from a suspicious IP, which could indicate a cyberattack. Without flow logs, you would have no visibility into what is happening at your network perimeter, it would be like the security guard keeping no records at all. For IT professionals, especially those studying for cloud certifications, understanding NSG flow logs is essential for designing secure, auditable cloud networks.

## Technical definition

NSG flow logs are a feature of Azure Network Watcher that provides detailed records of ingress and egress IP traffic through a Network Security Group (NSG). An NSG is a stateful packet filtering firewall that operates at OSI layers 3 and 4, controlling traffic based on rules defined with source/destination IP addresses, ports, and protocols (TCP, UDP, or any). When an NSG is associated with a subnet or a network interface (NIC), every packet traversing that boundary is evaluated against the NSG's rules. Flow logs capture key metadata for each flow: source IP, destination IP, source port, destination port, protocol, traffic direction (inbound or outbound), decision (allow or deny), and the timestamp. They also include the flow state (e.g., flow started, ongoing, ended) and the number of packets and bytes transferred.

The flow log data is aggregated into log files and written to a specified Azure Storage account in JSON format, typically in intervals of one minute for the first log, then recurring every 10 minutes by default (configurable). The logs are partitioned by storage account, container, date, and hour. Each flow record is identified by a tuple of five parameters (source IP, destination IP, source port, destination port, protocol), ensuring uniqueness. NSG flow logs support both version 1 and version 2 schema. Version 2 adds flow state information (B for beginning, C for continuing, E for end), which helps in reconstructing the lifetime of a connection.

Under the hood, Azure Network Watcher uses packet capture agents distributed across Azure fabric to collect flow data. The logs are not real-time; there is a processing delay of up to 10 minutes. The data is sampled at a rate of one record per flow per minute, so high-volume flows may have multiple records. Flow logs are crucial for compliance audits, they provide evidence of network traffic patterns and rule effectiveness. They also feed into Azure Sentinel and third-party SIEM systems for advanced threat detection. A common implementation involves enabling flow logs on critical NSGs (e.g., those protecting production servers) and generating alerts when denied traffic spikes, indicating reconnaissance or brute force attempts. Understanding NSG flow logs is required for the Azure Administrator (AZ-104) and Azure Security Engineer (AZ-500) exams, where candidates must know how to enable, interpret, and use flow logs for troubleshooting and security analysis.

## Real-life example

Think of a busy airport security checkpoint. The security officers check every passenger's boarding pass and ID before letting them through to the gates. They also have a list of banned items and individuals. Every interaction, whether the passenger is allowed through or denied, is recorded in a log. That log shows who the passenger was, which gate they were heading to, when they passed, and if they were turned away and why. The airport managers review this log daily to spot suspicious patterns, like a person trying to enter several times within an hour, or a sudden increase in denied entries from a particular airline.

NSG flow logs work exactly like this airport security log. In an Azure cloud environment, your virtual machines and services are like the airport gates. The Network Security Group is the security checkpoint. Each time a data packet tries to enter or leave your network, the NSG evaluates it against its rules. If the packet matches an allow rule, it passes through; if it matches a deny rule, it is blocked. NSG flow logs capture every such event, recording the source IP (the passenger), destination IP (the gate), port (the specific door), and whether the traffic was allowed or denied. Just as airport managers use the log to detect security threats, cloud administrators use flow logs to identify unusual traffic patterns, such as a distributed denial-of-service (DDoS) attack or an unauthorized scan. Without these logs, you would be flying blind, you would have no record of who is knocking at your network door or whether they got in.

## Why it matters

In modern cloud environments, network visibility is not a luxury, it is a necessity. NSG flow logs provide that visibility by recording every traffic decision made by a Network Security Group. This matters because security incidents often begin with network reconnaissance, where attackers probe for open ports or unprotected resources. Without flow logs, you would not know that such probing occurred until it is too late. For example, if a virtual machine starts receiving inbound traffic from an unusual geographic region on port 3389 (RDP), flow logs can alert you to a potential brute force attack. You can then take action by creating a deny rule or using Azure Security Center recommendations.

Flow logs also play a critical role in compliance. Regulations like GDPR, HIPAA, or PCI DSS often require evidence of network access controls and monitoring. By enabling flow logs and storing them in a tamper-proof Azure Storage account, you can demonstrate that you are actively monitoring network traffic and that your NSG rules are being enforced. Flow logs help with troubleshooting connectivity issues. If a user cannot reach an application, flow logs can show whether the traffic was blocked by an NSG rule, allowing you to quickly identify the problem rather than guessing. In a large enterprise, this can reduce downtime and improve incident response times. For IT professionals, especially those aiming for cloud certifications, understanding why flow logs matter is foundational: it separates reactive security from proactive security.

## Why it matters in exams

NSG flow logs appear prominently in several Microsoft Azure certification exams, particularly the Azure Administrator Associate (AZ-104) and the Azure Security Engineer Associate (AZ-500). In the AZ-104 exam, flow logs fall under the 'Manage Azure security' domain, specifically within 'Monitor and troubleshoot network resources'. Candidates are expected to understand how to enable flow logs using the Azure portal, PowerShell, or CLI, and how to retrieve and analyze log data from Azure Storage. A common exam objective is to 'Configure and manage NSG flow logs', which includes setting up a retention policy, configuring logging intervals, and integrating with Network Watcher.

In the AZ-500 exam, flow logs are tied to 'Implement network security' and 'Manage security operations'. The exam may ask you to interpret flow log entries to identify allowed vs. denied traffic, or to determine whether a particular NSG rule is effective. You might be given a scenario where an application is unreachable, and you must use flow logs to find the blocked traffic. Certification questions often test your ability to differentiate between NSG flow logs (which log at the NSG level) and Azure Monitor logs (which log at the resource level). Another common topic is the flow log retention policy: flow logs are stored in Azure Storage, and you must know how to set a retention period in days or use a policy to delete old logs to save costs.

For the more advanced Azure Network Engineer Associate (AZ-700) exam, flow logs are part of 'Design and implement secure network connectivity'. The exam may require you to design a logging solution using flow logs, including sending logs to Log Analytics workspaces or event hubs for real-time analysis. Candidates should also understand how flow logs can be used with traffic analytics (a separate service) to visualize traffic patterns. The bottom line: exam questions will test both conceptual understanding and practical configuration skills. Knowing the schema of a flow log, the difference between version 1 and version 2, and the typical delay (up to 10 minutes) are all fair game. Memorizing these details can earn you easy points.

## How it appears in exam questions

Exam questions about NSG flow logs usually fall into three categories: scenario-based, configuration, and troubleshooting. In scenario-based questions, you are given a situation like: 'Your company has a web application that frequently times out for remote users. The application runs on Azure VMs behind an NSG. You need to identify whether the traffic is being blocked. What should you do?' The correct answer often involves enabling NSG flow logs and analyzing the logs for denied traffic. Another pattern: 'You suspect a brute force attack on your Windows VM. Which log would show repeated failed RDP connection attempts from multiple IPs?' Again, NSG flow logs are the answer because they record every inbound connection attempt regardless of whether it succeeded or failed at the application level.

Configuration questions ask you to enable flow logs using the Azure portal or scripts. A typical question: 'You need to enable NSG flow logs on NSG1 with a retention policy of 30 days. The logs should be stored in a storage account named logsacct. Which Azure PowerShell cmdlet should you use?' The answer is 'Set-AzNetworkWatcherConfigFlowLog'. Or a multiple-choice question: 'Which service is required to work with NSG flow logs?' Answer: 'Azure Network Watcher'. Questions may also test your knowledge of the log format, such as whether flow logs include the packet contents (they do not; they only capture metadata).

Troubleshooting questions present a problem: 'After enabling NSG flow logs, you notice that the logs are not appearing in the storage account. What is the most likely cause?' Possible answers: the storage account is in a different region than the NSG; flow logs must be in the same region; Network Watcher is not enabled; the NSG is not associated with any subnet or NIC. Another common trap: 'Flow logs show denied traffic from an IP, but the NSG has an allow rule for that IP. Why is the traffic still blocked?' The answer could be that the NSG rule is at a lower priority, or there is another NSG at the subnet level that denies the traffic. Understanding these patterns helps you answer quickly and accurately.

## Example scenario

You are a cloud administrator for a company that runs an e-commerce website on Azure. The website consists of two virtual machines, a front-end web server and a backend database server. The web server is in a subnet with an NSG that allows inbound HTTP (port 80) and HTTPS (port 443) from the internet. The database subnet's NSG only allows inbound traffic from the web server subnet on port 1433 (SQL). Customers start reporting that the website is slow, and some pages are not loading. You suspect a network issue.

You decide to enable NSG flow logs on both subnet NSGs. After a few minutes, you open the logs stored in your Azure Storage account. You filter the logs for the web server NSG and see that many packets from an unusual IP address (203.0.113.50) are being allowed on port 8080, a port that is not typically used for web traffic. You review the NSG rules and realize that a junior admin had accidentally added a rule allowing port 8080 from any source. Meanwhile, the logs for the database NSG show that the web server's traffic on port 1433 is occasionally being denied. Looking closer, you see that the denied traffic comes from a specific IP within the web server subnet that does not belong to the web server. You investigate and discover that a developer had set up a test VM in the same subnet, and that test VM was trying to connect to the database but was blocked because its IP was not in the allowed list.

Using NSG flow logs, you were able to identify the misconfiguration on the web server NSG (the open port 8080) and also find the unauthorized database access attempt. You promptly fix the rules: remove the port 8080 rule and add the test VM's IP to the database NSG's allow list if needed. This scenario illustrates how flow logs reveal both security gaps and configuration errors that could go unnoticed without proper monitoring. It also shows that flow logs are not just for security, they are a powerful troubleshooting tool for everyday network issues.

## Common mistakes

- **Mistake:** Thinking NSG flow logs capture the actual packet contents.
  - Why it is wrong: NSG flow logs only capture metadata-source IP, destination IP, port, protocol, decision, and timestamp. They do not log the payload data inside the packets. To see packet contents, you need a packet capture tool like Azure Network Watcher's packet capture feature.
  - Fix: Remember: flow logs = metadata (who, what, when, allowed/denied). Packet capture = full packet contents. Use the right tool for the job.
- **Mistake:** Enabling flow logs on an NSG that is not associated with any subnet or NIC and expecting logs to appear.
  - Why it is wrong: Flow logs only generate traffic records for flows that actually pass through the NSG. If the NSG is not linked to a subnet or network interface, there is no traffic to log. The logs will be empty or not generated at all.
  - Fix: Always verify that the NSG is associated with a subnet or NIC before enabling flow logs. You can check this in the NSG overview page.
- **Mistake:** Believing flow logs are real-time.
  - Why it is wrong: Flow logs are not real-time. There is a processing delay of up to 10 minutes before the logs appear in the storage account. The logs are also aggregated in one-minute intervals, so you cannot see every individual packet in real time.
  - Fix: For real-time monitoring, use other tools like Azure Monitor alerts on metrics, or set up Traffic Analytics (which analyzes flow logs but still has a delay). Plan your incident response accordingly.
- **Mistake:** Assuming flow logs show application-level protocol details like HTTP request URLs.
  - Why it is wrong: Flow logs operate at OSI layers 3 and 4 (IP and TCP/UDP). They see the protocol (e.g., TCP) and port (e.g., 80), but not the application layer data. So they cannot show whether a request was an HTTP GET or POST.
  - Fix: Use application-level monitoring (like Application Insights or IIS logs) for details about HTTP requests. Flow logs are for network-level traffic visibility.
- **Mistake:** Forgetting to set a retention policy on the storage account, causing unlimited log accumulation and high costs.
  - Why it is wrong: By default, flow logs have no retention limit, they accumulate indefinitely in the storage account. This can lead to significant storage costs and compliance issues if logs are kept longer than needed.
  - Fix: When enabling flow logs, always specify a retention period in days (e.g., 30, 90, 365) directly in the flow log configuration. Azure will automatically delete logs older than that period.

## Exam trap

{"trap":"The exam asks: 'Which Azure service should you use to view real-time network traffic through an NSG?' and 'NSG flow logs' is listed as an option.","why_learners_choose_it":"Learners remember that flow logs provide network traffic information, so they assume it is real-time. The word 'real-time' is often emphasized in training, but flow logs are not real-time, they have a delay.","how_to_avoid_it":"Remember that NSG flow logs have a delay of up to 10 minutes. For real-time monitoring, you need Azure Network Watcher's connection monitor or packet capture. On the exam, if the question says 'real-time', look for 'connection monitor' or 'packet capture', not 'flow logs'."}

## Commonly confused with

- **NSG flow logs vs Azure Network Watcher packet capture:** Packet capture captures the actual packet data (including payload) for analysis, while NSG flow logs only capture metadata about the flow. Packet capture is an on-demand tool, whereas flow logs run continuously. Packet capture is used for deep forensic analysis, while flow logs are for ongoing monitoring. (Example: If you need to see the exact SQL query sent to a database, use packet capture. If you only need to know that traffic was allowed on port 1433, use flow logs.)
- **NSG flow logs vs Azure Monitor logs (diagnostic settings):** Azure Monitor logs can collect metrics and diagnostic data from Azure resources, but they are not specific to NSG traffic. NSG flow logs are a specialized subset of logging focused on network flows through an NSG. Diagnostic settings can send flow logs to Log Analytics, but they are not the same as the raw flow log data stored in storage accounts. (Example: Think of Azure Monitor logs as a general-purpose journal for all sorts of Azure events, while NSG flow logs are a dedicated ledger for network traffic only.)
- **NSG flow logs vs Traffic Analytics:** Traffic Analytics is an add-on that processes NSG flow logs and provides visualizations, top talkers, and security insights. It is not a logging feature itself; it consumes flow logs. Many people confuse the two, but traffic analytics is a separate paid service that runs on top of flow logs. (Example: NSG flow logs are the raw data (like a list of car license plates at a gate). Traffic Analytics is the tool that counts how many cars came from each city and highlights unusual patterns.)

## Step-by-step breakdown

1. **Enable Network Watcher** — Before you can use NSG flow logs, Azure Network Watcher must be enabled in the region where your NSG resides. Network Watcher is a regional service that provides monitoring and diagnostic tools for your virtual network. In most exams, it is enabled by default, but you should know how to enable it if needed.
2. **Select the NSG** — Choose the Network Security Group for which you want to enable flow logs. The NSG must be associated with at least one subnet or network interface to generate traffic. If it is not associated, you can still enable flow logs, but they will be empty. This step is crucial, flow logs are per-NSG, not per-subnet or per-VM.
3. **Configure storage account** — Flow logs are written to an Azure Storage account as JSON blobs. You must specify the storage account (or create a new one). The storage account should be in the same region as the NSG for optimal performance. You also set a retention period (in days) to automatically delete old logs. This is important for cost management and compliance.
4. **Enable flow logs (version 2 recommended)** — In the Azure portal, under the NSG's 'Monitoring' section, you can toggle 'Flow logs' to 'On'. You can also use PowerShell (Set-AzNetworkWatcherConfigFlowLog) or Azure CLI. Choose version 2 if available, as it includes flow state information (B, C, E) which is useful for reconstructing connection lifetimes. Version 1 is simpler but lacks that detail.
5. **Verify and access logs** — After enabling, allow up to 10 minutes for the first log file to appear. The logs are stored in the storage account under the container 'insights-logs-networksecuritygroupflowevent'. They are organized by date and hour. You can download them directly from the Azure portal or use tools like Azure Storage Explorer. You can also stream them to Log Analytics or Event Hubs for advanced analysis.
6. **Interpret log entries** — Each flow log entry contains fields like time, sourceIP, destinationIP, sourcePort, destinationPort, protocol, direction, action (Allowed/Denied), flowState (B/C/E), and packets/bytes transferred. For example, an entry with action=Denied and flowState=B indicates the first packet of a blocked flow. Analyzing these entries helps you identify which rules are being triggered and by whom.

## Practical mini-lesson

In a real-world cloud environment, enabling NSG flow logs is only the first step. Professionals need to know how to use those logs effectively. Here is a deeper look into what it takes to be proficient.

First, understand the log format thoroughly. Each flow log record is a JSON array containing multiple records. A typical record looks like: {"time":"2023-10-01T12:00:00.000Z", "systemId":"abc-123", "category":"NetworkSecurityGroupFlowEvent", "resourceId":"/subscriptions/.../NSG1", "operationName":"NetworkSecurityGroupFlowEvent", "properties":{"Version":2, "flows": [...]}}. The 'flows' array contains nested objects with 'flowTuples', which are comma-separated strings: source IP, destination IP, source port, destination port, protocol, direction, action, flow state, packets, bytes. The order matters. For example, a tuple: "10.0.0.4,52.170.0.1,443,80,T,O,B,10,1000" means: from 10.0.0.4 on port 443 to 52.170.0.1 on port 80, TCP, outbound (O), allowed (A? Actually 'T' means allowed? No, careful: in the tuple, action is 'A' for allowed or 'D' for denied. The example above is made up, the actual schema uses 'A' or 'D'. So 'T' is the protocol (TCP). So actually: 10.0.0.4,52.170.0.1,443,80,T,O,A,B,10,1000 would be TCP from port 443 to 80, outbound, allowed, beginning of flow, 10 packets, 1000 bytes. You need to memorize this order for exams.

Second, consider cost. Flow logs are not free, they incur storage costs for the log files. You pay for the storage account usage. A high-traffic NSG can generate gigabytes of logs per day. Professionals set a retention period (e.g., 30 days) and archive older logs to cold storage if needed for compliance. Some organizations use lifecycle management policies to automatically move logs to cool or archive tiers.

Third, security professionals often combine flow logs with Azure Sentinel. By sending flow logs to a Log Analytics workspace, you can create analytic rules that trigger alerts on suspicious patterns, like a single IP scanning multiple ports in a short time. For example, a rule might fire when a flow log entry shows 'Denied' traffic from the same source IP to more than 10 different destination ports within 5 minutes. This is how real attacks like port scanning are detected.

Finally, a common pitfall: enabling flow logs on every NSG without planning. This can overwhelm your storage and make analysis impractical. Instead, prioritize NSGs that control critical resources, such as those facing the internet or hosting sensitive databases. This selective approach saves costs and focuses monitoring efforts. NSG flow logs are a powerful tool, but they require thoughtful configuration and ongoing management to be truly useful in a production environment.

## Memory tip

Think of 'FLOw logs' as 'Flow, Location, Outcome': Flow metadata, Location (source/dest IP/port), Outcome (allowed/denied). Also remember: FLO = Facts (metadata only), no payload.

## FAQ

**Can I use NSG flow logs to see which user initiated a network connection?**

No. NSG flow logs only show IP addresses and ports, not user identities. To correlate a user with a connection, you would need to combine flow logs with application logs or Active Directory sign-in logs.

**Do I need Network Watcher enabled to use NSG flow logs?**

Yes. NSG flow logs are a feature of Azure Network Watcher. The service must be enabled in the region where your NSG resides. It is usually enabled by default in modern Azure subscriptions.

**How long does it take for NSG flow logs to appear after enabling?**

The first log file may appear within 1 minute, but subsequent logs are generated every 10 minutes. There is a processing delay, so logs are not real-time. Plan for up to 10 minutes delay.

**Can I send NSG flow logs to a Log Analytics workspace instead of a storage account?**

Yes. You can configure a diagnostic setting on the NSG to stream flow logs to a Log Analytics workspace, an Event Hub, or a storage account. However, the native flow log configuration in Network Watcher uses a storage account as the primary destination.

**Are NSG flow logs supported for all Azure regions?**

Yes, NSG flow logs are available in all Azure public regions, including sovereign clouds (Azure Government, Azure China) with some limitations. Check the Azure documentation for region-specific availability.

**What is the difference between version 1 and version 2 of NSG flow logs?**

Version 1 includes basic flow metadata (source/dest IP, port, protocol, action, packets, bytes). Version 2 adds the 'flowState' field (B for beginning, C for continuing, E for end), which helps reconstruct the duration of a connection. Version 2 is recommended.

## Summary

NSG flow logs are a critical monitoring feature in Azure that records IP traffic flowing through a Network Security Group. They capture metadata like source and destination IPs, ports, protocols, and whether the traffic was allowed or denied. These logs are stored in Azure Storage and can be analyzed for security auditing, troubleshooting connectivity issues, and compliance reporting. The key takeaway for IT certification learners is that flow logs are not real-time, do not capture packet payloads, and require Network Watcher to be enabled. They are tested extensively in Azure exams like AZ-104, AZ-500, and AZ-700.

Common mistakes include confusing flow logs with packet capture, assuming they are real-time, and forgetting to set a retention policy. Exam questions often present scenarios where you need to enable flow logs to diagnose blocked traffic or detect unauthorized access. Remember that flow logs are for network-layer visibility only, for application-layer details, you need other tools. By mastering NSG flow logs, you gain a powerful skill for both exam success and real-world cloud administration. Always think of them as your network's security camera, recording who passes through the gate and whether they are allowed in, but never the conversations they have inside.

---

Practice questions and the full interactive page: https://courseiva.com/glossary/nsg-flow-logs