This chapter covers IPsec transport mode versus tunnel mode, two fundamental encapsulation methods used in IPsec VPNs. Understanding the difference is critical for the N10-009 exam, as objective 4.4 (Network Security) frequently tests your ability to select the appropriate mode for a given scenario. Approximately 5-7% of exam questions touch on IPsec modes, often in the context of site-to-site vs. remote-access VPNs. By the end of this chapter, you will be able to explain the packet structure, use cases, and configuration differences between transport and tunnel modes.
Jump to a section
Imagine a company needs to send sensitive documents between its headquarters and a remote office. They have two options: an armored car or a sealed envelope inside a regular postal van. In tunnel mode, the entire original package (including the address labels) is placed inside a new, secure armored car with new external addresses. The armored car protects the entire original package, including the original sender and recipient information, from being read or tampered with. Anyone on the road sees only the armored car's origin and destination (the VPN gateways). In transport mode, only the document itself is sealed in a tamper-evident envelope, but the original postal address labels remain visible on the outside. The envelope protects the content, but a snooper can see that the package is going from HQ's mailroom to the remote office's mailroom (the original IP addresses). The envelope is then placed inside a regular postal van that uses the same route. The armored car (tunnel mode) provides more comprehensive protection by hiding the original addresses, while the sealed envelope (transport mode) protects only the payload but exposes the endpoints. The choice depends on whether you need to hide the internal network topology or just secure the data.
What IPsec Transport and Tunnel Modes Are
IPsec (Internet Protocol Security) is a suite of protocols defined by the IETF to provide secure communication at the IP layer. It can operate in two modes: transport mode and tunnel mode. The mode determines which part of the original IP packet is protected and how the IPsec headers are inserted.
Transport Mode: IPsec protects only the payload (the data portion) of the original IP packet. The original IP header remains intact and is not encrypted or authenticated. This mode is typically used for end-to-end communication between two hosts (e.g., a client and a server) or between a host and a security gateway.
Tunnel Mode: IPsec encapsulates the entire original IP packet (header and payload) inside a new IP packet. The new outer IP header contains the addresses of the IPsec endpoints (e.g., VPN gateways), while the original inner IP header remains protected. This mode is commonly used for site-to-site VPNs and remote-access VPNs where the traffic must traverse an untrusted network.
How IPsec Modes Work Internally
To understand the difference, we must examine the packet structure after IPsec processing. Assume we are using the ESP (Encapsulating Security Payload) protocol, which provides confidentiality (encryption) and optionally authentication.
#### Transport Mode with ESP
Original packet: [IP Header | TCP Header | Data]
After ESP transport mode:
[IP Header | ESP Header | TCP Header | Data | ESP Trailer | ESP Auth]
The original IP header is not encrypted. It contains the original source and destination IP addresses.
The ESP header is inserted after the IP header and before the transport layer header.
The TCP header, data, and ESP trailer are encrypted.
The ESP authentication trailer covers the ESP header, the encrypted payload, and the ESP trailer, but not the IP header (unless combined with AH).
#### Tunnel Mode with ESP
Original packet: [IP Header | TCP Header | Data]
After ESP tunnel mode:
[New IP Header | ESP Header | Original IP Header | TCP Header | Data | ESP Trailer | ESP Auth]
A new outer IP header is added. Its source and destination are the IPsec endpoints (e.g., VPN gateways).
The entire original IP packet (including its header) is encrypted and placed after the ESP header.
The ESP authentication trailer covers the ESP header, the encrypted original packet, and the ESP trailer.
The outer IP header is not encrypted.
Key Components, Values, and Defaults
Security Association (SA): An IPsec SA is a one-way logical connection between two IPsec peers. Each SA defines the mode (transport or tunnel), protocols (ESP, AH, or both), encryption algorithm, authentication algorithm, and keys. IKE (Internet Key Exchange) establishes SAs.
Encapsulation Security Payload (ESP): Provides encryption (confidentiality) and optional authentication. ESP uses an SPI (Security Parameter Index) in its header to identify the SA. Default encryption algorithms: AES (128, 192, 256 bits), 3DES (deprecated). Default authentication: HMAC-SHA1, HMAC-SHA256.
Authentication Header (AH): Provides authentication and integrity but no encryption. AH authenticates the entire packet, including the IP header (except mutable fields like TTL). In transport mode, AH protects the original IP header; in tunnel mode, it protects the outer IP header. AH is rarely used today because ESP can also provide authentication.
IKE (Internet Key Exchange): IKEv1 and IKEv2 negotiate SAs. IKE uses UDP port 500 and (for NAT traversal) UDP port 4500. Default IKE lifetime: 86400 seconds (24 hours) for Phase 1, 28800 seconds (8 hours) for Phase 2. These can be configured.
NAT Traversal (NAT-T): When IPsec packets traverse a NAT device, the NAT may modify the IP addresses, breaking AH integrity or causing issues with ESP. NAT-T encapsulates IPsec in UDP (port 4500) to avoid this. NAT-T detection occurs during IKE negotiation.
MTU Considerations: Tunnel mode adds 20-50+ bytes (depending on encryption and authentication) to the packet. This can cause fragmentation if the original packet is near the MTU. Path MTU discovery (PMTUD) should be enabled, or the MTU should be reduced (e.g., set TCP MSS to 1360 bytes for a typical 1500-byte MTU).
Configuration and Verification Commands
On Cisco IOS, IPsec is configured using crypto maps or tunnel interfaces. Below are examples for site-to-site VPN using tunnel mode.
#### Configuring Tunnel Mode (Site-to-Site)
! Define ISAKMP policy (IKE Phase 1)
crypto isakmp policy 10
encryption aes 256
authentication pre-share
group 14
lifetime 86400
crypto isakmp key mysecret address 203.0.113.2
! Define IPsec transform set (Phase 2)
crypto ipsec transform-set AES256-SHA esp-aes 256 esp-sha-hmac
mode tunnel
! Define crypto map
crypto map MYMAP 10 ipsec-isakmp
set peer 203.0.113.2
set transform-set AES256-SHA
match address 101
! Apply crypto map to interface
interface GigabitEthernet0/0
crypto map MYMAP
! Access control list to define interesting traffic
access-list 101 permit ip 10.0.1.0 0.0.0.255 10.0.2.0 0.0.0.255#### Configuring Transport Mode (Host-to-Host)
Transport mode is typically configured similarly, but the transform set is set to mode transport:
crypto ipsec transform-set AES256-SHA-TRANS esp-aes 256 esp-sha-hmac
mode transport#### Verification Commands
show crypto isakmp sa – displays IKE Phase 1 SAs.
show crypto ipsec sa – displays IPsec Phase 2 SAs, including encapsulation mode (tunnel or transport), SPI, and packet counts.
debug crypto ipsec – shows detailed IPsec negotiation.
ping with extended options to test VPN connectivity.
Interaction with Related Technologies
NAT: IPsec and NAT have inherent conflicts because NAT changes IP addresses, which can break AH integrity and cause ESP to fail if the NAT device does not support IPsec pass-through. NAT-T (UDP encapsulation) resolves this by adding a UDP header that NAT can translate without modifying the inner IPsec packet.
QoS: In tunnel mode, the outer IP header's DSCP field can be copied from the inner header or set explicitly. QoS policies must be applied to the outer tunnel traffic.
Routing: In tunnel mode, the inner IP addresses are not visible to the network between the gateways. Routing must direct traffic to the tunnel endpoints. In transport mode, the original IP addresses are visible, so routing can be based on them.
Multicast: IPsec does not natively support multicast. For multicast over VPN, GRE (Generic Routing Encapsulation) over IPsec is often used (tunnel mode).
Determine IPsec Mode Requirement
First, identify whether the VPN is host-to-host, host-to-gateway, or gateway-to-gateway. For host-to-host (e.g., two servers communicating securely), transport mode is appropriate because both endpoints are the IPsec peers and the original IP addresses are the same as the IPsec endpoints. For gateway-to-gateway (site-to-site), tunnel mode is required because the gateways act as proxies for the hosts behind them. The exam expects you to match the mode to the topology. A common trap: assuming transport mode can be used for site-to-site VPNs. It cannot, because the original source/destination IPs would be the hosts, not the gateways, causing routing and security issues.
Negotiate IKE Phase 1
The IPsec peers begin by establishing an IKE Phase 1 SA (also called ISAKMP SA). This uses UDP port 500. They authenticate using pre-shared keys, certificates, or other methods. They also agree on encryption (e.g., AES-256), authentication (e.g., SHA-256), Diffie-Hellman group (e.g., group 14), and a lifetime (default 86400 seconds). This creates a secure channel for further negotiation. During this phase, the peers also detect if NAT-T is needed by checking if either peer's IP address changes (NAT detection). If NAT is detected, they switch to UDP port 4500.
Negotiate IKE Phase 2
Using the secure channel from Phase 1, the peers negotiate one or more IPsec SAs (Phase 2). They agree on the IPsec protocol (ESP or AH), the mode (transport or tunnel), encryption and authentication algorithms (e.g., esp-aes 256 esp-sha-hmac), and a lifetime (default 28800 seconds or 100 MB of traffic). They also define the traffic selectors (interesting traffic) that will be protected, typically using ACLs. The result is a pair of unidirectional SAs (one for each direction). The SPI is generated and exchanged.
Apply IPsec to Outbound Packet
When a host sends a packet that matches the interesting traffic ACL, the IPsec stack processes it. In transport mode, the original IP header is preserved, and the ESP header is inserted after it. The payload (TCP/UDP header + data) is encrypted. In tunnel mode, the entire original IP packet is encrypted and encapsulated inside a new IP header with the gateway addresses. The packet's total length increases. The IPsec stack then calculates the ESP authentication data (ICV) and appends it. The packet is then forwarded to the destination gateway (tunnel) or host (transport).
Receive and Decapsulate Inbound Packet
The receiving peer inspects the outer IP header (tunnel) or the original IP header (transport) to determine that the packet is IPsec. It uses the SPI in the ESP header to look up the correct SA. It verifies the authentication data (ICV) to ensure integrity and authenticity. Then it decrypts the encrypted portion. In tunnel mode, it strips the outer IP header and ESP header, revealing the original inner IP packet. In transport mode, it strips the ESP header and trailer, leaving the original IP header intact. The decrypted packet is then forwarded to the final destination (host or next hop).
Scenario 1: Site-to-Site VPN between branch offices
A retail chain with 50 stores connects each store's network (192.168.x.0/24) to the headquarters (10.0.0.0/16) via IPsec tunnel mode. The VPN gateways are Cisco ISR routers at each location. The problem: each store's internal IP addresses overlap (all use 192.168.1.0/24). Tunnel mode solves this by allowing NAT (or dynamic routing) to handle overlapping subnets because the inner IP addresses are hidden inside the tunnel. The configuration uses IKEv2 with pre-shared keys, AES-256, SHA-256, and DH group 14. The MTU on the WAN interface is set to 1400 bytes to avoid fragmentation. Common misconfiguration: forgetting to adjust the MSS clamping (e.g., ip tcp adjust-mss 1360) on the tunnel interface, causing TCP connections to stall due to fragmentation. Performance: each tunnel handles up to 100 Mbps; with 50 tunnels, the head-end router must have sufficient CPU for encryption. If the router is underpowered, hardware crypto acceleration (e.g., Cisco CUBE) is needed.
Scenario 2: Remote access VPN for teleworkers Employees connect from home using a software client (e.g., Cisco AnyConnect) to the corporate ASA firewall. This uses IPsec tunnel mode (or SSL VPN, but IPsec is still common). The client's real IP address (e.g., 203.0.113.5) is hidden inside the tunnel; the corporate network sees only the VPN pool address (e.g., 10.10.10.10). Transport mode would not work here because the client's real IP would be exposed, and the ASA would not be able to route the decrypted packet to the internal host without knowing the inner IP. The ASA uses split tunneling to allow internet traffic to bypass the VPN for efficiency. Common issue: if split tunneling is misconfigured, all traffic goes through the VPN, causing high latency and bandwidth usage. Also, NAT-T must be enabled because the client is often behind a home router that performs NAT.
Scenario 3: Host-to-host IPsec for management traffic Two data center servers (10.1.1.1 and 10.2.2.2) need encrypted communication for database replication. Since they are directly reachable (no gateway acting as proxy), transport mode is used. The servers run Linux with strongSwan. Configuration uses ESP transport mode with AES-256-GCM for authenticated encryption. The advantage: lower overhead (no extra IP header) and simpler routing because the original IP addresses are used. However, if the servers are in different subnets behind routers, the routers must be able to route the original IPs. If a NAT device is between them, transport mode may break unless NAT-T is used. This scenario is less common in production because most server-to-server encryption is done at the application layer (TLS), but IPsec transport mode provides network-layer transparency.
N10-009 Objective 4.4 (Network Security) – IPsec Modes The exam expects you to:
Differentiate between transport and tunnel mode based on packet structure and use case.
Identify which mode is appropriate for site-to-site, remote-access, and host-to-host VPNs.
Understand that tunnel mode adds a new IP header, while transport mode does not.
Know that ESP encrypts the payload (transport) or the entire original packet (tunnel).
Recognize that AH authenticates the entire packet in transport mode (including IP header) but is rarely used.
Common Wrong Answers and Why Candidates Choose Them: 1. "Transport mode is used for site-to-site VPNs because it's more efficient." – This is wrong because site-to-site VPNs require the original host IPs to be hidden; transport mode exposes them. Candidates confuse efficiency with suitability. 2. "Tunnel mode encrypts only the payload." – This is the opposite. They mix up the modes. Remember: tunnel encrypts the whole original packet. 3. "AH provides encryption." – AH does not encrypt; it only authenticates. Candidates assume "authentication" includes encryption. 4. "Transport mode can be used with NAT without any special configuration." – Transport mode also needs NAT-T if NAT is present, but candidates think only tunnel mode has issues with NAT.
Specific Numbers and Terms to Memorize:
- UDP ports: IKE uses 500, NAT-T uses 4500.
- Default lifetimes: IKE Phase 1 = 86400 seconds (24h), Phase 2 = 28800 seconds (8h) or 100 MB.
- Common transform: esp-aes 256 esp-sha-hmac.
- DH groups: group 14 (2048-bit) is common; group 5 (1536-bit) is deprecated.
Edge Cases: - Traffic selectors: The exam may test that in tunnel mode, the traffic selectors are between the original source/destination subnets, not the gateway IPs. - NAT-T detection: Occurs during IKE Phase 1 when the peer's IP address in the IKE packet differs from the source IP (due to NAT). - AH vs. ESP: AH is rarely used; ESP is preferred because it provides encryption and optional authentication.
How to Eliminate Wrong Answers: - If a question mentions "gateway" or "site-to-site," eliminate transport mode. - If a question mentions "encrypting the original IP header," it must be tunnel mode. - If a question says "less overhead" or "lower latency," it points to transport mode. - Look for keywords: "end-to-end" often indicates transport mode; "VPN concentrator" or "remote access" indicates tunnel mode.
Transport mode encrypts only the payload; tunnel mode encrypts the entire original IP packet.
Tunnel mode adds a new outer IP header; transport mode does not.
Tunnel mode is used for site-to-site and remote-access VPNs; transport mode is used for host-to-host VPNs.
ESP provides encryption and optional authentication; AH provides only authentication (no encryption).
IKE uses UDP port 500; NAT-T uses UDP port 4500.
Default IKE Phase 1 lifetime is 86400 seconds; Phase 2 lifetime is 28800 seconds or 100 MB.
AH authenticates immutable fields of the IP header; ESP does not authenticate the outer IP header in transport mode.
NAT-T is required when IPsec packets traverse a NAT device, regardless of mode.
In tunnel mode, the inner IP header can have overlapping address spaces (e.g., same subnet at different sites).
The exam expects you to match the mode to the topology: gateway-to-gateway = tunnel; host-to-host = transport.
These come up on the exam all the time. Here's how to tell them apart.
Transport Mode
Protects only the payload (transport layer and above) of the original IP packet.
Original IP header remains intact and unencrypted.
Typically used for host-to-host or host-to-gateway (end-to-end) VPNs.
Less overhead (no extra IP header) – adds ~20-30 bytes for ESP.
Original source and destination IP addresses are visible in the packet.
Tunnel Mode
Protects the entire original IP packet (header and payload) by encapsulating it.
A new outer IP header is added with gateway addresses.
Typically used for site-to-site or remote-access VPNs (gateway-to-gateway or host-to-gateway).
More overhead (extra IP header + ESP) – adds ~40-60 bytes.
Original source and destination IP addresses are hidden inside the encrypted payload.
Mistake
IPsec tunnel mode encrypts the entire packet, including the new outer IP header.
Correct
The outer IP header is never encrypted; it must remain readable for routing. Only the original inner IP packet (header and payload) is encrypted. The outer header contains the source and destination IP addresses of the IPsec gateways.
Mistake
Transport mode is always more efficient than tunnel mode because it has less overhead.
Correct
While transport mode adds less overhead (no extra IP header), it is not always more efficient because it exposes the original IP addresses, which may cause security or routing issues. Efficiency gains are negligible compared to the overhead of encryption.
Mistake
AH provides encryption in addition to authentication.
Correct
AH only provides authentication and integrity, not encryption. The IPsec protocol that provides encryption is ESP. AH authenticates the entire packet (including immutable fields of the IP header), but the payload remains readable.
Mistake
IPsec transport mode cannot be used when NAT is present.
Correct
Transport mode can be used with NAT if NAT-T (UDP encapsulation) is enabled. NAT-T encapsulates the IPsec packet in UDP, allowing NAT devices to translate the outer UDP header without affecting the IPsec integrity. Both transport and tunnel modes can use NAT-T.
Mistake
Tunnel mode is only used for site-to-site VPNs, and transport mode is only used for host-to-host VPNs.
Correct
While these are the most common use cases, tunnel mode can also be used for host-to-gateway (remote access) VPNs, and transport mode can be used for host-to-gateway if the host is the IPsec peer. The mode is determined by what part of the packet is protected, not by the topology alone.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
Transport mode encrypts only the payload (the data portion) of the original IP packet, leaving the original IP header intact. Tunnel mode encrypts the entire original IP packet (header and payload) and encapsulates it inside a new IP packet with a new header. Transport mode is used for end-to-end communication between two hosts, while tunnel mode is used for site-to-site VPNs where the gateways act as proxies.
Use transport mode when the two IPsec peers are also the endpoints of the communication (e.g., two servers communicating directly). Transport mode is more efficient because it adds less overhead. Do not use transport mode for site-to-site VPNs because the original IP addresses would be exposed, defeating the purpose of hiding internal network topology.
No, the outer IP header is never encrypted. It must remain in plaintext for routers to forward the packet. Only the original inner IP packet (including its header) is encrypted and protected. The outer header contains the source and destination IP addresses of the VPN gateways.
Yes, but only if NAT-T (NAT Traversal) is enabled. Without NAT-T, the IPsec packet may be corrupted by NAT because the IP address in the packet (e.g., AH authenticated fields) changes. NAT-T encapsulates the IPsec packet in UDP (port 4500), allowing NAT to translate the UDP header safely. Both transport and tunnel modes can use NAT-T.
ESP (Encapsulating Security Payload) provides encryption (confidentiality) and optionally authentication. AH (Authentication Header) provides only authentication and integrity, but no encryption. ESP is more commonly used because it offers confidentiality. AH authenticates the entire packet, including immutable fields of the IP header, while ESP does not authenticate the outer IP header in transport mode.
In a site-to-site VPN, the VPN gateways (routers/firewalls) are the IPsec peers, not the end hosts. Tunnel mode encapsulates the original packet from the host, hiding the internal IP addresses. This allows multiple sites to use overlapping IP subnets and ensures that the internal network structure is not exposed to the internet. Transport mode would expose the host IPs, which is undesirable.
Transport mode adds approximately 20-30 bytes (ESP header/trailer + auth). Tunnel mode adds approximately 40-60 bytes (new IP header + ESP). The exact overhead depends on the encryption algorithm and authentication method. For example, AES-256 with SHA-256 adds 22 bytes for ESP plus 20 bytes for the new IP header in tunnel mode.
You've just covered IPsec Transport vs Tunnel Mode — now see how well it sticks with free N10-009 practice questions. Full explanations included, no account needed.
Done with this chapter?