This chapter covers the two primary networking models for Azure Kubernetes Service (AKS): Azure CNI and Kubenet. Understanding the differences between these models is critical for the AZ-104 exam, as questions on AKS networking appear in approximately 10-15% of exam questions, often as scenario-based decisions. You will learn the internal packet flow, IP address management, performance characteristics, and configuration options for each model, along with when to choose one over the other.
Jump to a section
Imagine a city with two transportation models for delivering packages between buildings. In the first model (Kubenet), every building has a single main address, and inside each building, rooms are identified by internal extension numbers. A delivery truck (node) arrives at the building's main address, and a central concierge (kube-proxy and IPtables) inside receives the package. The concierge checks a directory and hands the package to the correct internal room (pod). However, the package's return address shows only the building's main address, not the room. For a package to leave the building, the concierge rewrites the 'from' address to the building's main address (NAT). This means the outside world never sees individual rooms, and all inter-building traffic must go through the concierge, which can become a bottleneck. In the second model (Azure CNI), every room in every building gets its own unique street address directly from the city's address registry (Azure VNet). A package can be delivered directly from any room to any other room, regardless of building, because each room has a globally routable address. No address rewriting is needed. The concierge still exists but only for service discovery, not for traffic forwarding. The trade-off: the city must pre-allocate a large block of addresses to each building, and if a building runs out of addresses, no new rooms can be added until the city assigns more. The first model conserves addresses but adds overhead; the second model provides direct connectivity but consumes more address space.
What Are Azure CNI and Kubenet?
Azure Kubernetes Service (AKS) supports two primary networking plugins: Kubenet (also known as basic networking) and Azure CNI (Container Networking Interface, also known as advanced networking). Both plugins enable pod-to-pod communication, pod-to-service communication, and external connectivity, but they differ fundamentally in how they assign IP addresses and route traffic.
Kubenet is the default networking plugin for AKS clusters created via the Azure CLI or portal without specifying a networking option. It provides a logical overlay network where pods receive IP addresses from a private, node-local address space that is not directly routable on the Azure virtual network (VNet). Pod traffic must traverse the node's kernel networking stack, where IP forwarding and Network Address Translation (NAT) are applied.
Azure CNI is the advanced networking plugin that integrates pods directly into the Azure VNet. Each pod receives an IP address from the subnet assigned to the node's virtual network interface (vNIC). Pods are directly reachable from other pods, services, and on-premises networks without NAT, because their IPs are native to the VNet.
How Kubenet Works Internally
When a pod in a Kubenet cluster sends a packet to another pod on a different node, the following steps occur:
Pod-to-Node: The pod's network namespace sends the packet via the node's bridge (typically cbr0). The packet has a source IP from the pod's private CIDR (e.g., 10.244.0.0/16) and a destination IP of the target pod.
Node Routing: The node's kernel checks its routing table. Because the destination IP belongs to the pod CIDR but is not on the local node, the packet is forwarded to the node's default gateway (the Azure VNet subnet gateway) after being NATed.
NAT (Source IP Translation): The node performs source NAT (SNAT) on the packet, changing the source IP from the pod IP to the node's primary IP address on the Azure VNet subnet. This is necessary because the pod IP is not routable on the VNet.
Forwarding to Destination Node: The packet traverses the Azure VNet to the destination node's IP. The destination node receives the packet and, based on its routing table, forwards it to the target pod after applying destination NAT (DNAT) if needed.
Return Traffic: The return packet follows the reverse path, with the source node performing reverse NAT to restore the original pod IP.
Kubenet uses iptables rules to manage NAT and packet filtering. Each node runs a kube-proxy component that programs iptables rules for services. The number of iptables rules grows linearly with the number of services, which can impact performance at scale (typically beyond 1,000 services).
How Azure CNI Works Internally
With Azure CNI, the packet flow is simpler:
Pod-to-Node: The pod's network namespace sends the packet directly to the node's virtual network interface. The source IP is the pod's VNet IP (e.g., 10.0.1.5), and the destination IP is the target pod's VNet IP (e.g., 10.0.2.10).
Node Routing: The node's kernel checks its routing table. Because the destination IP is in the same VNet subnet (or a different subnet), the packet is forwarded directly to the Azure virtual router without NAT.
Azure VNet Routing: Azure's software-defined network (SDN) routes the packet to the destination node's vNIC based on the destination IP address.
Delivery: The destination node receives the packet and delivers it to the target pod's network namespace via the Azure CNI bridge (e.g., azure0).
No NAT is performed, so the original pod IP is preserved end-to-end. This enables direct connectivity to on-premises networks via VPN or ExpressRoute, and simplifies network policies.
Key Components, Values, and Defaults
Kubenet Defaults: - Pod CIDR: 10.244.0.0/16 (configurable) - Node subnet: Provided by the user (e.g., 10.0.0.0/24) - Max pods per node: 110 by default (configurable up to 250) - IP addresses consumed: Each node uses 1 IP from the VNet subnet for the node itself. Pod IPs come from the private pod CIDR, which is not in the VNet. - Network policy: Supported via Calico (third-party) or Azure Network Policy (preview).
Azure CNI Defaults: - Pod IPs: Assigned from the same subnet as the node (e.g., 10.0.0.0/24). Each pod consumes one IP from that subnet. - Max pods per node: 30 by default (configurable up to 250, but limited by subnet size). - IP addresses consumed: Each node uses 1 IP from the VNet subnet for the node itself, plus one IP per pod. This means a /24 subnet (256 IPs) can support at most 256 - 5 (Azure reserved) - N (nodes) pods. - Network policy: Built-in support via Azure Network Policy (using Azure Firewall or Calico). - Performance: Higher throughput and lower latency because no NAT is involved.
Configuration and Verification Commands
Creating a Kubenet cluster:
az aks create --resource-group myRG --name myAKSCluster --network-plugin kubenetCreating an Azure CNI cluster:
az aks create --resource-group myRG --name myAKSCluster --network-plugin azure --vnet-subnet-id /subscriptions/.../subnets/mySubnetVerifying the network plugin:
az aks show --resource-group myRG --name myAKSCluster --query networkProfile.networkPluginChecking pod IPs:
kubectl get pods -o wideInspecting node routes (Kubenet):
kubectl exec -it <node-name> -- ip routeInteraction with Related Technologies
Azure Firewall: In Kubenet, outbound traffic from pods goes through the node's SNAT. To force traffic through Azure Firewall, you must use custom routes (UDRs) pointing to the firewall as the next hop. In Azure CNI, you can apply UDRs directly to the pod subnet.
Network Security Groups (NSGs): With Azure CNI, NSGs applied to the pod subnet filter traffic to/from pods directly. With Kubenet, NSGs on the node subnet see traffic from node IPs, not pod IPs, so filtering is less granular.
Service Mesh: Azure CNI is recommended for service meshes like Istio because they require direct pod-to-pod communication without NAT.
Calico: Both plugins support Calico for network policies, but Azure CNI can also use Azure Network Policy.
Performance Considerations
Latency: Azure CNI has lower latency because packets do not undergo NAT. Kubenet adds 5-10% overhead per packet due to NAT processing.
Throughput: Azure CNI can achieve line-rate throughput (up to 40 Gbps for certain VM sizes) because packets bypass the host networking stack. Kubenet is limited by the node's NAT capacity.
Scalability: Kubenet scales to larger clusters because it conserves VNet IP addresses. Azure CNI may exhaust subnet IPs quickly; for large clusters, use large subnets (e.g., /16) or multiple subnets.
Security Implications
Pod Identity: In Azure CNI, pods have their own VNet IPs, making them directly identifiable and subject to NSG rules. In Kubenet, pods are hidden behind node IPs, which can complicate auditing and network policy enforcement.
eBPF: Azure CNI can leverage eBPF for faster packet processing (Azure CNI powered by Cilium). Kubenet relies on iptables, which is slower to update with many rules.
When to Use Each
Choose Kubenet when: - You need to conserve VNet IP addresses (e.g., limited address space). - You have a small cluster (< 50 nodes) and low pod density (< 20 pods/node). - You do not require direct pod connectivity from on-premises. - You are okay with slightly higher latency.
Choose Azure CNI when: - You need direct pod-to-pod connectivity without NAT. - You require on-premises connectivity to pods (via VPN/ExpressRoute). - You need granular NSG rules based on pod IPs. - You are using a service mesh or need high performance. - You have sufficient IP address space (e.g., /16 subnet).
Migration Considerations
You cannot migrate a cluster from Kubenet to Azure CNI in-place. You must create a new cluster with Azure CNI and migrate workloads. Plan IP address allocation carefully before deployment.
Pod Sends Packet to Another Pod
A pod in a Kubenet cluster generates an IP packet destined for another pod on a different node. The source IP is a private pod IP (e.g., 10.244.0.5), and the destination is another pod IP (e.g., 10.244.1.10). The packet leaves the pod's network namespace and enters the node's root network namespace via the virtual Ethernet pair (veth). The node's bridge (cbr0) examines the destination MAC address. Since the destination is not on the local bridge, the packet is forwarded to the node's routing stack. The kernel consults the routing table: it sees that the destination belongs to the pod CIDR but is not directly connected, so it routes the packet to the default gateway (the node's primary IP on the Azure subnet) after applying source NAT.
Node Performs Source NAT (SNAT)
The node's iptables rules (managed by kube-proxy) intercept the packet. A MASQUERADE rule changes the source IP from the pod IP (10.244.0.5) to the node's primary IP (e.g., 10.0.0.4). The destination IP remains unchanged. The node also rewrites the source port to a unique ephemeral port to track the connection. This NAT is necessary because the pod IP is not routable on the Azure VNet. The packet is then sent out through the node's physical NIC to the Azure VNet. The node maintains a conntrack entry to reverse the NAT on return traffic.
Packet Traverses Azure VNet
The packet now has source IP 10.0.0.4 (node) and destination IP 10.0.1.6 (another node). Azure's software-defined network routes the packet based on the destination IP. If the destination node is in the same subnet, the packet is delivered directly via the Azure virtual switch. If in a different subnet, it goes through the Azure virtual router. No additional NAT occurs. The packet reaches the destination node's NIC. The destination node's kernel receives the packet and checks its routing table: it sees that the destination IP (10.0.1.6) is assigned to its own NIC, so it accepts the packet.
Destination Node Delivers to Pod
The destination node's kernel looks up the destination IP in its local routing table. It finds that the IP belongs to a pod on the node. The node's bridge (cbr0) forwards the packet to the correct veth pair leading to the target pod. The pod receives the packet with the original source IP (10.0.0.4, the source node's IP) because the source pod's IP was NATed. The destination pod sends a response back to 10.0.0.4. The destination node's iptables rules (if any) may apply DNAT to map services, but for direct pod-to-pod, no DNAT is applied. The return packet travels to the source node, which performs reverse NAT to map the destination IP back to the original pod IP (10.244.0.5).
Azure CNI Pod Sends Packet
In Azure CNI, the pod sends a packet with source IP 10.0.1.5 (pod's VNet IP) and destination IP 10.0.2.10 (another pod's VNet IP). The packet goes through the veth pair to the node's root namespace. The node's Azure CNI bridge (azure0) forwards the packet to the node's routing stack. The kernel checks the routing table: the destination IP is in the same VNet (e.g., 10.0.0.0/16), so it is forwarded directly to the Azure virtual router without NAT. The packet is sent out the node's NIC with the original source IP. Azure routes the packet to the destination node's NIC. The destination node delivers the packet to the target pod via its own azure0 bridge. The entire path preserves the original pod IPs, allowing direct connectivity.
Enterprise Scenario 1: Microservices with On-Premises Integration
A large financial services company runs a microservices application on AKS that needs to communicate with an on-premises database over ExpressRoute. They choose Azure CNI because pods must be directly reachable from on-premises without NAT. The cluster uses a /16 subnet (65,536 IPs) to accommodate 500 nodes with 30 pods each (15,000 IPs). They configure Azure Network Policy to restrict pod-to-pod traffic based on application labels. The network team monitors IP utilization and sets alerts when subnet usage exceeds 80%. A common misconfiguration is forgetting to reserve IPs for Azure management (5 IPs per subnet) and for node scaling. During a scale-up event, if the subnet is full, new nodes fail to join. The solution is to use multiple subnets per node pool or overprovision address space.
Enterprise Scenario 2: Cost-Sensitive Dev/Test Environment
A startup uses Kubenet for their non-production AKS clusters to save VNet IP addresses. They have a /24 subnet (251 usable IPs) shared across multiple clusters. Each cluster has 3 nodes with 110 pods each (max pods per node set to 110). Since pod IPs come from a private 10.244.0.0/16 CIDR, they do not consume VNet IPs. The downside is that they cannot use NSGs to filter pod traffic; they rely on Calico network policies. Performance is acceptable for low-traffic workloads. A problem arises when they need to connect to an Azure SQL Database: outbound traffic from pods uses SNAT, so the database sees the node IP, not the pod IP. This complicates auditing. They work around this by using Azure SQL firewall rules that allow the node subnet, but security teams are unhappy. The team plans to migrate to Azure CNI when the application goes to production.
Scenario 3: High-Performance Computing with AKS
A media streaming company runs a real-time video transcoding service on AKS. They require low latency and high throughput. They choose Azure CNI with Cilium eBPF for accelerated networking. Each node is a D32s v3 VM with 32 vCPUs and 128 GB RAM. They set max pods per node to 250, but the subnet is a /20 (4096 IPs) to support 16 nodes. They use Azure Accelerated Networking (SR-IOV) on the VMs to achieve 40 Gbps throughput. Performance monitoring shows that Azure CNI adds less than 1% overhead compared to bare-metal networking. A common issue is that when using Kubernetes services of type LoadBalancer, the external traffic policy must be set to 'local' to preserve client IP, which requires careful routing configuration.
What AZ-104 Tests on AKS Networking
The AZ-104 exam focuses on your ability to choose between Kubenet and Azure CNI based on requirements. Objective 3.3 (Deploy and manage Azure Kubernetes Service) includes sub-objectives on networking configuration. Expect 2-3 questions that present a scenario and ask which networking plugin to use, or ask about IP address consumption, or about connectivity to on-premises.
Common Wrong Answers and Why Candidates Choose Them
Wrong Answer 1: 'Choose Kubenet when you need direct pod connectivity from on-premises.' Why wrong: Kubenet uses NAT, so pod IPs are hidden. On-premises cannot route to pod IPs directly. Azure CNI is required.
Wrong Answer 2: 'Azure CNI uses fewer IP addresses than Kubenet.' Why wrong: Azure CNI consumes one VNet IP per pod, while Kubenet uses a separate private CIDR. For large clusters, Kubenet conserves VNet IPs.
Wrong Answer 3: 'Kubenet supports network policies by default.' Why wrong: Kubenet does not include built-in network policy; you must install a third-party plugin like Calico.
Wrong Answer 4: 'You can migrate a cluster from Kubenet to Azure CNI without downtime.' Why wrong: The networking plugin is set at cluster creation time. Migration requires creating a new cluster.
Specific Numbers and Terms to Memorize
Default max pods per node: Kubenet = 110, Azure CNI = 30 (configurable up to 250).
Azure reserves 5 IPs per subnet (first 4 and last 1).
Pod CIDR for Kubenet: 10.244.0.0/16 (default).
Command to check network plugin: az aks show --query networkProfile.networkPlugin.
Azure CNI requires a subnet with enough IPs for all pods + nodes + Azure reserved.
Edge Cases and Exceptions
If you set --max-pods higher than the subnet capacity in Azure CNI, node creation fails.
Kubenet can use Azure Network Policy in preview, but the exam treats it as requiring Calico.
For clusters with Windows node pools, Azure CNI is required (Kubenet does not support Windows).
When using Azure CNI with Calico, you must set --network-policy calico at creation time.
How to Eliminate Wrong Answers
If the scenario mentions 'direct connectivity from on-premises', eliminate Kubenet.
If the scenario mentions 'limited IP address space', eliminate Azure CNI.
If the scenario mentions 'Windows nodes', eliminate Kubenet.
If the scenario mentions 'high performance' or 'low latency', choose Azure CNI.
If the scenario mentions 'cost savings on IP addresses', choose Kubenet.
Kubenet is the default AKS networking plugin; it uses a private pod CIDR and SNAT for outbound traffic.
Azure CNI assigns pod IPs from the VNet subnet, enabling direct connectivity without NAT.
Kubenet default max pods per node is 110; Azure CNI default is 30.
Azure CNI requires sufficient VNet IP addresses for all pods; plan subnet size accordingly.
Kubenet is suitable for small clusters or when IP address space is limited.
Azure CNI is required for Windows node pools, direct on-premises connectivity, and high performance.
Network plugin cannot be changed after cluster creation; migrate workloads to a new cluster.
Azure reserves 5 IPs per subnet for management (first 4 and last 1).
These come up on the exam all the time. Here's how to tell them apart.
Kubenet
Pods get IPs from a private, non-VNet CIDR (e.g., 10.244.0.0/16).
Uses SNAT for outbound traffic; pod IPs are hidden.
Default max pods per node: 110 (configurable up to 250).
Conserves VNet IP addresses (only node IPs consume VNet space).
Slightly higher latency due to NAT processing.
Azure CNI
Pods get IPs directly from the VNet subnet.
No NAT; pod IPs are directly routable.
Default max pods per node: 30 (configurable up to 250, subject to subnet size).
Consumes one VNet IP per pod; requires large subnets for big clusters.
Lower latency and higher throughput; supports up to 40 Gbps with accelerated networking.
Mistake
Kubenet does not support network policies at all.
Correct
Kubenet supports network policies via third-party plugins like Calico. Azure CNI has built-in support for Azure Network Policy and also supports Calico.
Mistake
Azure CNI pods cannot communicate across subnets.
Correct
Azure CNI pods can communicate across subnets within the same VNet because Azure routes traffic between subnets. They can also communicate across peered VNets.
Mistake
Kubenet is always slower than Azure CNI.
Correct
Kubenet has higher latency due to NAT, but for small clusters with low traffic, the difference is negligible. Azure CNI is faster but uses more IP addresses.
Mistake
You can change the network plugin after cluster creation.
Correct
The network plugin is selected at cluster creation time and cannot be changed. You must create a new cluster to switch plugins.
Mistake
Azure CNI requires a separate subnet for pods.
Correct
Azure CNI assigns pod IPs from the same subnet as the node. You can use a dedicated subnet for pods, but it is not required. The subnet must have enough IPs for both nodes and pods.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
No. Kubenet does not support Windows node pools. You must use Azure CNI if you need Windows containers. This is a common exam trap: if the scenario mentions Windows, eliminate Kubenet immediately.
Run the Azure CLI command: `az aks show --resource-group <rg> --name <cluster> --query networkProfile.networkPlugin`. The output will be either 'kubenet' or 'azure'.
New nodes and pods cannot be created. Existing workloads continue to run, but scaling up fails. You must add a new subnet to the node pool or resize the existing subnet (requires downtime). Plan for at least 50% headroom.
Azure Network Policy is only supported with Azure CNI. For Kubenet, you can use Calico for network policies. The exam treats Kubenet as requiring a third-party policy engine.
Yes, both plugins support LoadBalancer services. However, with Kubenet, the external traffic policy must be set to 'Cluster' (default) to preserve client IP? Actually, with Kubenet, the client IP is lost because of SNAT. To preserve client IP, you need to set externalTrafficPolicy: Local, but this may cause uneven load distribution.
Azure reserves 5 IP addresses per subnet: the first four (x.x.x.0-3) and the last one (x.x.x.255). These cannot be assigned to resources. Remember this when calculating subnet capacity for Azure CNI.
No. The networking plugin is set at cluster creation time. You must create a new cluster with Azure CNI and migrate your workloads. This is a critical exam point.
You've just covered AKS Networking: Azure CNI vs Kubenet — now see how well it sticks with free AZ-104 practice questions. Full explanations included, no account needed.
Done with this chapter?