This chapter covers network virtualization and VXLAN (Virtual Extensible LAN), a key technology for modern data centers and cloud environments. For the N10-009 exam, understanding VXLAN is critical as it appears in questions about network segmentation, overlay networks, and data center architecture. Approximately 5-10% of exam questions touch on network virtualization concepts, with VXLAN being the primary example. You will need to know how VXLAN encapsulates Ethernet frames in UDP packets, the role of VTEPs, and how it differs from traditional VLANs.
Jump to a section
Imagine you run a logistics company that needs to move goods between two cities, but the roads are only wide enough for small vans. You have large shipping containers that can hold many boxes, but the containers are too big for the roads. So, you put each container onto a flatbed truck (the outer packet) that fits the road. The truck has its own license plate (outer IP header) and follows the road's traffic rules (routing). The container (inner packet) is sealed and its contents (original MAC addresses, VLAN tags) are invisible to the road. At the destination, the truck is removed, and the container is delivered to the local warehouse (VTEP). This allows you to move many small boxes together, securely, across a network that wasn't designed for large containers. In VXLAN, the truck is the UDP packet with VXLAN header, the container is the inner Ethernet frame, and the road is the IP network. The truck's route (outer IP) is based on the destination VTEP's IP, not the container's destination MAC. This decouples the virtual network from the physical network, enabling Layer 2 extension over Layer 3 infrastructure.
What is Network Virtualization?
Network virtualization is the process of abstracting network resources (switches, routers, firewalls) from the underlying physical hardware, creating logical or virtual networks that operate independently. This allows multiple virtual networks to share the same physical infrastructure while maintaining isolation and policy separation. The primary drivers are multi-tenancy (as in cloud providers) and the need to stretch Layer 2 domains across Layer 3 boundaries for workload mobility (e.g., live migration of VMs).
The Problem with VLANs
Traditional VLANs (IEEE 802.1Q) have a 12-bit VLAN ID field, limiting them to 4094 usable VLANs (0 and 4095 reserved). In large data centers with thousands of tenants, this is insufficient. Moreover, VLANs are Layer 2 constructs; they cannot span across IP routers without complex configurations like VPLS or OTV. VLANs also rely on the spanning tree protocol (STP) to prevent loops, which can block redundant links and reduce bandwidth utilization.
What is VXLAN?
VXLAN (Virtual Extensible LAN), defined in RFC 7348, is an overlay network technology that encapsulates Layer 2 Ethernet frames in Layer 3 UDP packets. It uses a 24-bit VXLAN Network Identifier (VNI), supporting up to 16 million unique segments (compared to 4094 VLANs). VXLAN is designed to run over existing IP networks, allowing virtual networks to span across Layer 3 boundaries without requiring changes to the physical network.
How VXLAN Works
VXLAN operates using VXLAN Tunnel Endpoints (VTEPs). A VTEP is a device (physical switch, virtual switch, or software) that performs encapsulation and decapsulation. When a VM sends an Ethernet frame, the source VTEP encapsulates the frame inside a VXLAN packet:
Outer Ethernet header: source/destination MAC of the VTEPs (or next-hop router)
Outer IP header: source IP (source VTEP), destination IP (destination VTEP or multicast group)
Outer UDP header: destination port 4789 (IANA assigned for VXLAN)
VXLAN header: 8 bytes including VNI (24 bits), reserved flags, and reserved fields
Inner Ethernet frame: original frame from VM (including original MAC addresses, VLAN tag if any)
The encapsulated packet is then forwarded over the IP network like any other UDP packet. At the destination VTEP, it decapsulates and delivers the original Ethernet frame to the destination VM.
VXLAN Packet Format Details
Outer MAC: Typically, the MAC address of the next-hop router or the destination VTEP's MAC if on same subnet.
Outer IP: Source IP is the source VTEP's IP (usually a loopback or management IP). Destination IP can be unicast (for known destinations) or multicast (for unknown/BUM traffic).
Outer UDP: Source port is a hash of the inner frame's header (for load balancing), destination port is 4789.
VXLAN Header: VNI (24 bits), flags (I flag set to 1 for VXLAN, others reserved), reserved bytes.
Inner Frame: Original Ethernet frame from the VM, including its own MAC addresses, VLAN tag (if present, stripped before encapsulation per RFC 7348, but optionally preserved via 'VXLAN with VLAN' or 'VXLAN-GBP').
VXLAN Modes: Unicast and Multicast
Unicast: Used when the destination VTEP is known. The source VTEP maintains a forwarding table mapping inner MAC addresses to remote VTEP IPs (learned via data plane or control plane). For known unicast frames, the packet is sent directly to the destination VTEP's IP.
Multicast: Used for broadcast, unknown unicast, and multicast (BUM) traffic. The VTEPs join a multicast group (one per VNI). When a VM sends a broadcast, the source VTEP encapsulates it with the multicast group IP as destination. All VTEPs in that group receive it and decapsulate locally. This reduces flooding compared to sending to all VTEPs individually.
VXLAN with EVPN (Ethernet VPN)
In modern data centers, VXLAN is often combined with EVPN (RFC 7432) as the control plane. EVPN uses BGP to distribute MAC address and VNI information between VTEPs, replacing data-plane learning. This provides better scalability, faster convergence, and integrated multi-tenancy. EVPN uses MP-BGP with address family L2VPN EVPN to advertise MAC/IP routes, IMET routes (for multicast), and more. The N10-009 exam may touch on EVPN as a control plane for VXLAN, but focus is on VXLAN itself.
VXLAN vs. VLAN
VLAN: 12-bit ID (4094 segments), Layer 2 only, limited scalability, requires STP.
VXLAN: 24-bit VNI (16 million segments), overlay over Layer 3, uses IP routing, no STP needed (or can use a separate underlay). VXLAN can carry VLAN tags inside the inner frame, allowing migration of VLAN-based networks.
VXLAN Configuration Example (Cisco Nexus)
interface nve1
source-interface loopback0
member vni 10000
mcast-group 239.1.1.1This creates a Network Virtualization Edge (NVE) interface, sets the source IP to loopback0, and associates VNI 10000 with multicast group 239.1.1.1. For EVPN, you would also configure BGP EVPN address family.
Verification Commands
show nve vni - display VNI information
show nve peers - show remote VTEPs
show mac address-table - see MAC addresses learned via VXLAN
show vxlan - summary of VXLAN tunnels
Interaction with Related Technologies
VXLAN can run over any IP network, including MPLS, GRE, or IPsec. It is commonly used with spine-leaf architectures (Clos networks). The underlay network provides IP connectivity between VTEPs; VXLAN provides the overlay.
VXLAN is often used with VMware NSX, Cisco ACI, or open-source solutions like Open vSwitch (OVS).
VXLAN does not require multicast in all cases; some implementations use a unicast-only mode with a control plane (e.g., EVPN) to distribute MAC information, so BUM traffic is handled via replication or ARP suppression.
VXLAN Performance Considerations
Encapsulation adds 50 bytes overhead (20 IP + 8 UDP + 8 VXLAN + 14 outer MAC = 50 bytes). This reduces the effective MTU for inner frames. The underlay must support an MTU of at least 1550 bytes (1500 + 50) or use jumbo frames (typically 9000 bytes).
Source UDP port hashing enables load balancing across ECMP paths in the underlay, as routers see different flows.
VXLAN is stateless; each packet is independent. Stateful services (firewalls) must be placed at the VTEP or in the overlay.
VXLAN and Security
VXLAN does not provide encryption. For security, use IPsec or MACsec on the underlay, or use VXLAN with IPsec (VXLAN over IPsec). Some vendors offer VXLAN with GENEVE (Generic Network Virtualization Encapsulation) that supports option headers for security metadata.
VNI provides segmentation, but not confidentiality. Tenants are isolated by VNI, but if an attacker gains access to the underlay, they could potentially inject traffic.
Common Exam Traps
Mistaking VXLAN for a Layer 2 technology that requires Layer 2 underlay. VXLAN actually runs over Layer 3 IP networks; the underlay is IP, not Ethernet.
Thinking VXLAN uses TCP. It uses UDP port 4789. Some older implementations used port 8472 (IETF draft), but 4789 is standard.
Confusing VNI with VLAN ID. VNI is 24-bit, VLAN is 12-bit. VXLAN can carry a VLAN tag inside the inner frame, but they are separate.
Believing VXLAN eliminates the need for STP in the overlay. STP is not needed within VXLAN segments because the overlay is a full mesh of VTEPs; however, the underlay may still use STP or, more commonly, use routed access with no STP.
VM sends Ethernet frame
A virtual machine (VM) on a hypervisor generates an Ethernet frame destined for another VM possibly on a different physical host. The frame includes source and destination MAC addresses, and optionally a VLAN tag. The hypervisor's virtual switch (vSwitch) or a hardware VTEP receives the frame. The vSwitch identifies the destination MAC and determines that it is not locally known (or is known to be on a remote VTEP). It then forwards the frame to the VTEP function (software or hardware) for encapsulation.
VTEP encapsulates frame in VXLAN
The source VTEP takes the original Ethernet frame and encapsulates it. It prepends a VXLAN header (8 bytes) containing the VNI (24-bit) and flags. Then it adds a UDP header (8 bytes) with source port computed from a hash of the inner frame (for ECMP) and destination port 4789. Next, an outer IP header (20 bytes) with source IP = VTEP's IP, destination IP = remote VTEP's IP (unicast) or multicast group. Finally, an outer Ethernet header (14 bytes) with source MAC = VTEP's MAC, destination MAC = next-hop router's MAC. The resulting packet is up to 50 bytes larger than the original.
Packet traverses underlay IP network
The encapsulated packet is forwarded by the physical network (switches and routers) based on the outer IP header. The underlay network treats it as a standard UDP packet. Routers perform ECMP load balancing based on the outer IP header and UDP source port (due to the hash). The packet may pass through multiple Layer 3 hops. The inner frame is completely opaque to the underlay; switches do not see the original MAC addresses or VLAN tags. The destination IP must be reachable via routing.
Destination VTEP receives and decapsulates
The destination VTEP receives the packet. It checks the destination UDP port (4789) to identify it as VXLAN. It verifies the VXLAN header and extracts the VNI. It then strips the outer headers (Ethernet, IP, UDP, VXLAN), leaving the original inner Ethernet frame. The VTEP uses the VNI to determine the correct virtual network (e.g., VXLAN segment). It then forwards the inner frame to the destination VM based on the inner destination MAC address.
VTEP learns MAC-to-VTEP mapping
The destination VTEP learns the source MAC address of the inner frame and associates it with the source VTEP's IP address. This mapping is stored in a forwarding table (usually in hardware TCAM). For data-plane learning, the VTEP uses the source IP of the outer packet. For control-plane learning (EVPN), the mapping is advertised via BGP. This learning is used for subsequent unicast frames: when a VM sends to that MAC, the VTEP knows to encapsulate with the correct remote VTEP IP.
VXLAN is widely deployed in enterprise data centers and cloud environments. Consider a large financial institution with multiple data centers across cities. They need to migrate virtual machines between data centers for disaster recovery without changing IP addresses. VLANs cannot span across Layer 3 boundaries, but VXLAN can. The institution deploys VXLAN with EVPN as the control plane. Each data center has a cluster of spine-leaf switches. VTEPs are configured on the leaf switches (ToR switches). The underlay is an IP fabric (e.g., OSPF or BGP). Each tenant gets a unique VNI. When a VM moves, its MAC address is advertised via BGP EVPN, and traffic is redirected to the new location. The network engineer must ensure the underlay MTU is set to at least 1550 bytes (commonly 9000 jumbo frames). A common misconfiguration is forgetting to increase the MTU on all underlay interfaces, causing packet drops for VXLAN traffic. Another scenario is a cloud provider hosting thousands of tenants. Using VLANs would limit them to 4094 tenants. With VXLAN, they assign each tenant a VNI, allowing millions of isolated networks. The provider uses a centralized control plane (e.g., VMware NSX) that manages VTEPs on hypervisors. Performance considerations: VXLAN encapsulation adds CPU overhead on software VTEPs, so hardware offload (e.g., on SmartNICs) is recommended for high throughput. Troubleshooting: When a VM cannot communicate with another VM across VXLAN, common issues include: (1) Underlay routing not reaching the remote VTEP IP, (2) MTU mismatch (inner packet too large), (3) VNI mismatch between VTEPs, (4) Firewall blocking UDP port 4789. The engineer would use 'ping' to test underlay connectivity, 'traceroute' to check path, and 'show nve peers' to verify VTEP adjacency. They might also capture packets to see if VXLAN encapsulation is occurring correctly.
The N10-009 exam tests VXLAN under Objective 1.7 (Explain basic network virtualization concepts). Specifically, you need to know: (1) What VXLAN is and why it is used (to overcome VLAN limitations), (2) The role of VTEPs, (3) The VNI (24-bit, 16 million segments), (4) UDP port 4789, (5) That VXLAN encapsulates Layer 2 frames in Layer 3 packets, (6) The difference between VXLAN and VLAN. Common wrong answers: (1) 'VXLAN uses TCP port 4789' – candidates confuse UDP with TCP; many protocols use TCP, but VXLAN uses UDP for stateless encapsulation. (2) 'VXLAN requires a Layer 2 underlay' – this is false; VXLAN runs over any IP network (Layer 3). (3) 'VXLAN replaces VLANs entirely' – VXLAN can coexist and carry VLAN tags inside. (4) 'VXLAN encrypts traffic' – it does not; encryption is separate. Exam questions often ask: 'Which technology allows Layer 2 extension over Layer 3 networks?' Answer: VXLAN. Or 'What is the maximum number of VXLAN segments?' 2^24 = 16,777,216. Memorize the VNI size and UDP port. Edge cases: VXLAN can use multicast for BUM traffic, but not all implementations require multicast (EVPN can use unicast). Also, some questions may ask about the overhead: 50 bytes (20 IP + 8 UDP + 8 VXLAN + 14 outer MAC). However, the outer MAC is often considered part of the underlay overhead; the encapsulation overhead from IP header onwards is 50 bytes. To eliminate wrong answers: if the answer mentions 'VLAN' as a solution for multi-tenancy across sites, it's likely wrong because VLANs are limited. If the answer says 'VXLAN uses TCP', eliminate it. If the answer says 'VXLAN provides encryption', eliminate it.
VXLAN (Virtual Extensible LAN) is defined in RFC 7348 and uses a 24-bit VNI for up to 16 million segments.
VXLAN encapsulates Layer 2 Ethernet frames in UDP packets (port 4789) over an IP underlay.
VTEP (VXLAN Tunnel Endpoint) performs encapsulation and decapsulation; can be hardware or software.
VXLAN can use multicast for BUM traffic or a control plane like EVPN for unicast-only operation.
Encapsulation adds 50 bytes overhead (20 IP + 8 UDP + 8 VXLAN + 14 outer MAC); underlay MTU must be increased accordingly.
VXLAN does not provide encryption; security must be added separately.
VXLAN enables workload mobility (VM migration) across Layer 3 boundaries without changing IP addresses.
Common exam mistake: confusing VXLAN with VLAN or thinking VXLAN uses TCP.
These come up on the exam all the time. Here's how to tell them apart.
VLAN (802.1Q)
12-bit VLAN ID (4094 usable)
Layer 2 only; cannot cross routers without additional protocols
Uses STP for loop prevention, blocking redundant links
Frame format adds 4-byte tag to Ethernet frame
Limited scalability for large multi-tenant environments
VXLAN
24-bit VNI (16 million segments)
Overlay over Layer 3 IP network; can span across routers
No STP needed in overlay; underlay uses IP routing (ECMP)
Encapsulation adds 50 bytes overhead (IP+UDP+VXLAN+MAC)
Designed for massive scalability in data centers and clouds
Mistake
VXLAN uses TCP port 4789 for encapsulation.
Correct
VXLAN uses UDP port 4789. The protocol is stateless and does not require the reliability of TCP.
Mistake
VXLAN requires a Layer 2 underlay network to function.
Correct
VXLAN is an overlay that runs over any IP network (Layer 3). The underlay can be routed, and VXLAN packets are forwarded based on outer IP headers.
Mistake
VXLAN can only support 4094 segments like VLANs.
Correct
VXLAN uses a 24-bit VNI, supporting up to 16,777,216 segments, far more than the 4094 VLANs.
Mistake
VXLAN provides encryption and security for the encapsulated traffic.
Correct
VXLAN does not include encryption. Security must be provided by other means such as IPsec or MACsec on the underlay.
Mistake
VXLAN eliminates the need for STP in the entire network.
Correct
VXLAN eliminates the need for STP within the overlay (since it's a full mesh), but the underlay network may still use STP or, more commonly, use routed access with no STP.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
VLAN (802.1Q) uses a 12-bit VLAN ID (4094 segments) and operates at Layer 2, requiring the same broadcast domain. VXLAN uses a 24-bit VNI (16 million segments) and encapsulates Ethernet frames in UDP over IP, allowing Layer 2 segments to span Layer 3 networks. VXLAN is more scalable and suitable for multi-tenant data centers.
VXLAN uses UDP destination port 4789 (IANA assigned). Some early implementations used port 8472, but the standard is 4789. The source port is typically a hash of the inner frame for load balancing.
Not necessarily. VXLAN can use multicast for BUM traffic (broadcast, unknown unicast, multicast). However, with a control plane like EVPN, you can use unicast for known traffic and handle BUM via replication or ARP suppression, eliminating the need for multicast in the underlay.
A VTEP (VXLAN Tunnel Endpoint) is a device that encapsulates and decapsulates VXLAN packets. It can be a physical switch (e.g., ToR switch), a virtual switch (e.g., VMware vDS), or a software function (e.g., Open vSwitch). The VTEP has an IP address used as the source/destination in the outer IP header.
In data-plane learning, the VTEP learns MAC-to-VTEP mappings by inspecting the source MAC of the inner frame and the source IP of the outer packet. In control-plane learning (e.g., EVPN), MAC addresses are advertised via BGP, providing more scalable and deterministic learning.
VXLAN adds 50 bytes of overhead: outer Ethernet header (14 bytes), outer IP header (20 bytes), outer UDP header (8 bytes), and VXLAN header (8 bytes). The inner frame is unchanged. Therefore, the underlay MTU must be at least 1550 bytes (1500 + 50) to avoid fragmentation.
Yes, the inner Ethernet frame can include a VLAN tag (802.1Q). The VTEP typically strips the VLAN tag before encapsulation (per RFC 7348), but the tag can be preserved in the inner frame if needed (e.g., using VXLAN with VLAN mode). The VNI itself provides segmentation, so the VLAN tag is often redundant.
You've just covered Network Virtualization and VXLAN — now see how well it sticks with free N10-009 practice questions. Full explanations included, no account needed.
Done with this chapter?