IPSec VPN

IPSec (IP Security) is a suite of protocols that provides confidentiality, integrity, authentication, and anti-replay protection for IP packets. It operates at Layer 3 of the OSI model, meaning it can protect any traffic that uses IP—TCP, UDP, ICMP, etc. Unlike SSL/TLS which only protects specific applications (like web or email), IPSec secures all traffic between two endpoints, making it ideal for site-to-site VPNs connecting entire networks.

The core problem IPSec solves is that the Internet is inherently insecure. Packets traverse multiple routers and links where they can be intercepted, modified, or replayed. IPSec ensures that even if a packet is captured, it cannot be read (encryption), altered (integrity), or replayed (anti-replay window).

IPSec has two main security protocols: - ESP (Encapsulating Security Payload): Provides encryption, integrity, authentication, and anti-replay. It is the most commonly used protocol. ESP can operate in transport mode (protects only the payload) or tunnel mode (protects the entire original packet). - AH (Authentication Header): Provides integrity and authentication but NOT encryption. It is rarely used because it does not provide confidentiality and has issues with NAT traversal.

The security associations (SAs) are the contracts that define how two peers protect their traffic. An SA is unidirectional—two SAs are needed for bidirectional communication. Each SA is identified by a Security Parameter Index (SPI) and contains parameters like encryption algorithm, hash algorithm, lifetime, and keys.

IKE is the protocol used to establish and manage SAs. It operates in two phases: - IKE Phase 1: Creates a secure, authenticated tunnel (ISAKMP SA) used to protect IKE Phase 2 negotiations. Phase 1 can use Main Mode (6 messages) or Aggressive Mode (3 messages, less secure). The peers authenticate using pre-shared keys, digital certificates, or other methods. They also negotiate the IKE policy (encryption, hash, DH group, lifetime). Default lifetime is 86400 seconds (1 day). - IKE Phase 2: Uses the secure Phase 1 tunnel to negotiate the IPSec SAs (one for each direction). This phase uses Quick Mode (3 messages). It negotiates the transform set (ESP/AH, encryption, hash, mode) and the traffic selectors (interesting traffic defined by ACLs). Default lifetime is 3600 seconds (1 hour).

To verify IPSec VPN status on a Cisco router, use the following commands:

show crypto isakmp sa

Example output:

IPv4 Crypto ISAKMP SA
dst             src             state          conn-id slot status
10.1.1.1        10.2.2.2        QM_IDLE           1001    0 ACTIVE

show crypto ipsec sa

Example output:

interface: GigabitEthernet0/0
    Crypto map tag: CMAP, local addr 10.1.1.1
   protected vrf: (none)
   local  ident (addr/mask/prot/port): (10.1.1.0/255.255.255.0/0/0)
   remote ident (addr/mask/prot/port): (10.2.2.0/255.255.255.0/0/0)
   current_peer 10.2.2.2 port 500
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 100, #pkts encrypt: 100, #pkts digest: 100
    #pkts decaps: 100, #pkts decrypt: 100, #pkts verify: 100
    #send errors 0, #recv errors 0

     local crypto endpt.: 10.1.1.1, remote crypto endpt.: 10.2.2.2
     path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/0
     current outbound spi: 0x12345678(305419896)
     PFS (Y/N): N, DH group: none

     inbound esp sas:
      spi: 0x87654321(2271560481)
        transform: esp-aes 256 esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 2001, flow_id: SW:1, sibling_flags 80000046, crypto map: CMAP
        sa timing: remaining key lifetime (k/sec): (4608000/2879)
        IV size: 16 bytes
        replay detection support: Y
        Status: ACTIVE

     outbound esp sas:
      spi: 0x12345678(305419896)
        transform: esp-aes 256 esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 2002, flow_id: SW:2, sibling_flags 80000046, crypto map: CMAP
        sa timing: remaining key lifetime (k/sec): (4608000/2879)
        IV size: 16 bytes
        replay detection support: Y
        Status: ACTIVE

Key fields: - #pkts encaps/encrypt/digest: number of packets sent through the VPN - #pkts decaps/decrypt/verify: number received - inbound esp sas and outbound esp sas: show the two unidirectional SAs - sa timing: remaining lifetime in kilobytes and seconds - transform: the encryption and hash algorithms - Status: ACTIVE if SA is up

show crypto ipsec transform-set

Shows configured transform sets.

show crypto map

Shows crypto map configuration and which interface it is applied to.

In enterprise networks, IPSec VPNs are deployed in several common scenarios:

1. Site-to-Site VPN between Branch and Headquarters: A company with a main office (10.1.0.0/16) and multiple branch offices (each with a /24 subnet) uses IPSec tunnel mode over the Internet. Each branch router has a crypto map pointing to the HQ router's public IP. The crypto ACL permits traffic between the branch LAN and the HQ LAN. This replaces expensive leased lines. Performance considerations: encryption/decryption is CPU-intensive; for high throughput, use hardware crypto acceleration (e.g., Cisco routers with onboard crypto engine) or dedicated VPN appliances. Misconfiguration: if the ACLs are not mirrored, traffic may be dropped or sent in the clear (if the ACL is too permissive).

2. Remote Access VPN with IPSec Client: Teleworkers use a software client (e.g., Cisco AnyConnect) to connect to the corporate network. The client initiates IKE with the head-end router/firewall. The head-end uses a crypto map with dynamic peers (no fixed peer IP). The client's traffic is encrypted and tunneled to the head-end, which decrypts and forwards to internal resources. Common issues: NAT traversal problems if the client is behind a home router that does not support IPSec passthrough; using UDP 4500 (NAT-T) solves this. Authentication is often via digital certificates or RADIUS.

3. DMVPN (Dynamic Multipoint VPN): For large-scale hub-and-spoke or spoke-to-spoke VPNs, DMVPN combines IPSec with mGRE (multipoint GRE) and NHRP. The hub has a single crypto map, and spokes register their public IPs with the hub via NHRP. Spokes can build direct IPSec tunnels between each other (spoke-to-spoke) without going through the hub, reducing latency. This is common in large retail or banking networks. Misconfiguration: incorrect NHRP settings or missing route injection can cause traffic to be dropped.

In all scenarios, network engineers must consider: (1) MTU issues—IPSec adds overhead (up to 58 bytes per packet), which may cause fragmentation; adjust TCP MSS or set the DF bit appropriately. (2) Routing—static routes pointing to the tunnel interface or dynamic routing over the VPN (e.g., OSPF over GRE over IPSec). (3) Monitoring—using SNMP to track SA counts, packet drops, and CPU load. (4) Redundancy—using multiple peers or backup crypto maps with different priorities.