What Is Underlay vs Overlay Network in Networking?

Imagine you are in a large office building with many floors. The building itself has physical hallways, staircases, and elevators. These are the underlay network. They exist in the real world and you can touch them. Now imagine the building also has a system of colored lines painted on the walls and floors that guide different types of people to their destinations. For example, blue lines for visitors, red lines for employees, and green lines for deliveries. These lines are like the overlay network. They use the same physical hallways and elevators, but they create separate paths that are easier to manage and change. If you need to change a visitor route, you do not need to knock down a wall or install a new elevator. You just repaint the line.

In computer networking, the underlay consists of the actual routers, switches, fiber optic cables, and wireless access points that send packets of data as electrical or light signals. The overlay network is a virtual network created using software. It encapsulates packets inside other packets so they can travel across the underlay in a way that is invisible to the physical devices. For instance, a company can create an overlay network that connects all its branch offices securely over the public internet, without needing to lease private physical circuits. The underlay internet might take different paths each time, but the overlay ensures the data arrives safely and securely. This separation makes networks more flexible, scalable, and easier to manage.

A simple way to grasp this is to think of the postal system. The underlay is the entire postal infrastructure: mail trucks, sorting centers, airplanes, and delivery vans. The overlay is a special courier service that uses the same postal trucks but puts your package in a sealed, tamper-proof bag with a unique tracking number. The courier service can reroute your package, provide extra security, and guarantee delivery times without changing how the postal trucks operate. In the same way, an overlay network can add features like encryption, traffic prioritization, or network segmentation without modifying the underlying hardware.

In networking, the underlay network refers to the physical infrastructure responsible for the actual forwarding of data packets. This includes routers, switches, optical transport equipment, and the cabling or wireless links that connect them. The underlay operates at Layers 1 through 3 of the OSI model, handling physical signaling, data link framing, and IP routing. Protocols such as OSPF, IS-IS, and BGP are commonly used in the underlay to establish reachability and exchange routing information. The underlay must provide connectivity and bandwidth, and it is often optimized for performance and reliability.

An overlay network is a logical or virtual network built on top of the underlay. It creates a separate, software-defined topology that can be independent of the physical topology. Overlay networks use encapsulation techniques to wrap packets from the overlay inside packets that the underlay can forward. Common encapsulation protocols include VXLAN (Virtual Extensible LAN), GRE (Generic Routing Encapsulation), IPsec, and MPLS (Multiprotocol Label Switching). For example, VXLAN encapsulates Layer 2 Ethernet frames inside UDP packets, allowing a virtual LAN to extend across Layer 3 boundaries. This enables network virtualization, where multiple virtual networks can share the same physical infrastructure without interference.

Overlay networks are a cornerstone of modern data center and enterprise architectures, particularly in Cisco's SD-Access and ACI (Application Centric Infrastructure) solutions. In SD-Access, the underlay is typically a routed fabric using IS-IS, while the overlay uses VXLAN with a control plane based on LISP. This separation allows network administrators to design and change virtual networks without reconfiguring physical devices. Overlay networks also enable multitenancy, where different customers or departments have isolated network segments over a shared underlay. In cloud environments, overlay networks like those in VMware NSX or Amazon Virtual Private Cloud (VPC) allow virtual machines to move between physical hosts while retaining their IP addresses and network policies.

One key technical aspect is the control plane versus data plane separation. In many overlay designs, the underlay handles the data plane, forwarding encapsulated packets based on the outer header. The overlay control plane manages the mapping between virtual endpoints and their underlay locations. For instance, in VXLAN EVPN (Ethernet VPN), BGP is used to distribute MAC address and IP address information across the overlay, so each endpoint knows where to send encapsulated traffic. This approach significantly improves scalability compared to traditional flooding-based learning.

Understanding the difference between underlay and overlay networks is critical for anyone working in modern IT, especially as networks become more software-driven. In real-world IT environments, overlay networks allow companies to create multiple isolated networks over a single physical fabric. This is essential in data centers where different applications, customers, or security zones need to coexist without conflict. For example, a hospital might use an overlay network to separate patient data traffic from guest Wi-Fi traffic, all while using the same physical switches and cables. This reduces hardware costs and simplifies management.

In enterprise networking, overlays enable seamless mobility. When a user moves from one office to another, the overlay network can keep their IP address and access policies consistent, even if the underlay topology changes. This is vital for voice and video applications that break if the network path changes abruptly. Overlay networks also facilitate secure remote connectivity. A company can build an overlay using IPsec VPNs to connect branch offices over the internet, creating a virtual private network that is isolated from the public underlay.

In cybersecurity, overlay networks support microsegmentation. Instead of placing a firewall at the network edge, overlay networks can enforce security policies between individual virtual machines within the same data center. For instance, a web server and a database server can be placed in separate overlay segments, and traffic between them is only allowed if the policy permits. This greatly reduces the attack surface. Without overlays, achieving this level of isolation would require physical firewalls and complex VLAN configurations that are difficult to scale.

From a career perspective, most Cisco CCNP exams, especially ENCOR, cover underlay and overlay concepts extensively. Professionals who master these concepts can design and troubleshoot complex networks using technologies like VXLAN, SD-Access, and EVPN. Employers look for engineers who understand how to build scalable, flexible networks that can adapt to changing business needs. Overlay networks are not just a theoretical idea; they are deployed in thousands of organizations worldwide, from small businesses to global enterprises.

Exam questions on underlay and overlay networks appear in several forms. The most common is the scenario-based multiple-choice question. For instance, the exam might describe a company that wants to extend a Layer 2 VLAN across three data centers that are connected by a routed Layer 3 underlay. The question asks which technology should be used. The correct answer is VXLAN, because it encapsulates Ethernet frames over IP, creating an overlay that operates over the routed underlay. A distractor might be to use a traditional VLAN trunk, which would require a Layer 2 underlay and is not scalable.

Another question pattern involves troubleshooting. A scenario describes a user unable to reach a server on a different subnet, even though both are part of the same overlay network. The question asks what to check first. The correct answer would be to verify the underlay routing between the VTEPs, because if the physical path is broken, the overlay cannot function. This type of question tests whether you understand that the overlay depends on the underlay for basic reachability.

There are also design questions. A company is planning a new data center with multiple tenants. The question asks whether to use a flat underlay with VLANs or an overlay with VXLAN. The correct answer is the overlay, because it supports 16 million virtual networks versus 4096 VLANs, and it allows tenant isolation without physical segmentation. These questions require you to know the limitations of traditional architectures and the benefits of overlays.

Configuration-related questions might ask you to match commands to their function. For example, you might be given a list of commands and asked to identify which one configures a VXLAN Network Identifier (VNI) on a switch. Or you might need to identify which protocol is used for the overlay control plane in SD-Access (LISP). Some questions test your ability to interpret output from show commands, like 'show vxlan tunnel' or 'show lisp session'. You must recognize the underlay source and destination IPs versus the overlay endpoint IDs.

Finally, there are compare-and-contrast questions: 'Which of the following is true about underlay networks compared to overlay networks?' Options might include statements about where routing decisions are made, which layer of the OSI model they operate at, or which one requires more manual configuration. Knowing these distinctions helps you eliminate wrong answers quickly.

In practice, underlay and overlay networks are designed together, but they are managed separately. As a network engineer, you first build the underlay to be simple, resilient, and high-performance. Common underlay designs use a spine-leaf topology in data centers, where every leaf switch connects to every spine switch, providing predictable latency and easy scalability. The underlay runs a dynamic routing protocol like BGP (with IPv4 unicast) or IS-IS, and it should have enough bandwidth to accommodate all overlay traffic plus overhead from encapsulation.

Once the underlay is stable, you deploy the overlay. The overlay requires careful configuration of VTEPs. In a Cisco environment, you might use Nexus 9000 switches with VXLAN EVPN. The overlay control plane uses BGP EVPN to exchange type-2 routes (MAC/IP) and type-3 routes (IMET for multicast). You must configure the VNI-to-VLAN mapping and ensure that the underlay MTU is large enough to handle the extra 50 bytes from VXLAN encapsulation. If the underlay MTU is 1500, you must either increase it to 1550 or enable IP fragmentation, which is best avoided.

Common issues in production include MTU mismatches: when a VTEP sends a 1550-byte packet over an underlay with 1500 MTU, the packet is fragmented or dropped. Engineers must configure the underlay interfaces with a larger MTU, typically 1550 or 1600, and ensure that all intermediate devices also support it. Another issue is misconfiguration of the VXLAN destination UDP port (default 4789). If firewall ACLs in the underlay block this port, the overlay fails. Always verify that UDP 4789 is permitted between VTEPs.

From a broader perspective, overlay networks are central to network automation. Tools like Ansible can push overlay configurations to hundreds of switches simultaneously. Cisco DNA Center automates the provisioning of SD-Access overlay fabrics. As a professional, you should learn to read overlay troubleshooting logs and correlate them with underlay metrics. For example, if overlay traffic is slow, check underlay link utilization and routing convergence. The overlay is only as good as the underlay that supports it.