Troubleshoot: ACL Blocking Legitimate Traffic

An Access Control List (ACL) is a sequential list of permit or deny statements that filter packets based on criteria such as source IP, destination IP, protocol, and port numbers. On Cisco IOS, ACLs are processed top-down: the router examines each packet against the first entry; if it matches, the action (permit/deny) is applied and no further entries are checked. If no match is found by the end of the list, the packet is implicitly denied. This implicit deny is the default behavior for all ACLs, and it is the most common reason legitimate traffic gets blocked — the traffic simply does not match any permit entry.

But there are many other reasons an ACL might block traffic that should be permitted. These include: - Incorrect order of entries: A deny entry that matches the traffic appears before a permit entry. Since processing stops at the first match, the traffic is denied. - Incorrect wildcard mask: A wildcard mask that is too broad or too narrow can cause the ACL to match unintended traffic or miss intended traffic. - Incorrect source/destination addresses: The ACL entry specifies the wrong IP address or subnet. - Incorrect protocol or port: For extended ACLs, specifying the wrong protocol (e.g., TCP vs UDP) or port number (e.g., 80 vs 8080). - Applied to the wrong interface or direction: The ACL is placed on the wrong interface, or the direction (in/out) is incorrect. - Missing entries for return traffic: For stateful traffic like TCP, the ACL must permit return packets (e.g., established connections). - Incorrect placement of 'established' keyword: Misunderstanding how the 'established' keyword works for TCP.

When a packet enters or exits an interface (depending on the ACL direction), the router performs the following steps:

To troubleshoot an ACL that blocks legitimate traffic, use these commands:

Example output:

Router# show access-lists 100
Extended IP access list 100
    10 permit tcp 10.0.0.0 0.0.0.255 any eq 80 (10 matches)
    20 deny ip any any (5 matches)

Here, entry 10 has 10 matches, meaning 10 packets were permitted. Entry 20 (the implicit deny) has 5 matches, meaning 5 packets were denied. If legitimate traffic is being denied, check the match counts: if the deny entry has matches, the traffic matches it. If no matches on any entry, the traffic is being dropped by implicit deny (or not reaching the interface).

Pattern 1: Deny before permit

access-list 100 deny ip 10.0.0.0 0.0.0.255 any
access-list 100 permit tcp 10.0.0.5 0.0.0.0 any eq 80

Traffic from 10.0.0.5 to port 80 will be denied because the first entry matches all traffic from the 10.0.0.0/24 subnet (including .5).

Pattern 2: Wrong wildcard mask

access-list 100 permit tcp 10.0.0.0 0.0.0.0 any eq 80

This permits only the single host 10.0.0.0 (which is a network address, not a host). It should be 0.0.0.255 for the subnet or host 10.0.0.5 for a specific host.

Pattern 3: Missing permit for return traffic For TCP, you need to permit return packets. Using the established keyword can help, but it only matches packets with the ACK or RST bit set. If you forget to permit return traffic, the three-way handshake completes (SYN, SYN-ACK, ACK), but data packets from the server to the client may be dropped if they are not ACK packets.

Pattern 4: ACL applied to wrong interface/direction An ACL applied inbound on interface Gig0/0 will filter traffic entering that interface. If you apply it outbound on the same interface, it filters traffic leaving. Applying it to the wrong interface altogether (e.g., Gig0/1 instead of Gig0/0) will not affect traffic on Gig0/0.

Troubleshooting ACL blocking legitimate traffic requires a systematic approach: verify the ACL entries and order, check the wildcard masks, confirm the interface and direction, and use show commands to see match counts. Remember the implicit deny and that order matters.

In a real enterprise network, ACLs are used to enforce security policies between departments, such as allowing HR to access the payroll server while blocking other departments. A common scenario: the HR subnet is 10.10.10.0/24, and the payroll server is at 10.10.20.100. The network engineer configures an extended ACL on the core router's interface facing HR:

access-list 101 permit tcp 10.10.10.0 0.0.0.255 host 10.10.20.100 eq 443
access-list 101 deny ip any any
interface GigabitEthernet0/1
 ip access-group 101 in

But HR users report they cannot access the payroll server. The engineer runs show access-lists 101 and sees zero matches on the permit entry but many matches on the implicit deny. After checking, they realize the payroll server actually uses port 8443, not 443. The fix is to change the port or permit both.

Another scenario: a company uses an ACL to block SSH access from the internet to their core router, but allow SSH from the management subnet (192.168.1.0/24). The ACL is applied inbound on the external interface:

access-list 102 permit tcp 192.168.1.0 0.0.0.255 any eq 22
access-list 102 deny tcp any any eq 22

However, the management station at 192.168.1.50 cannot SSH to the router's external IP. The engineer checks show ip interface and finds the ACL is applied outbound instead of inbound. After correcting the direction, SSH works.

Scale and performance: ACLs are processed in hardware on modern switches, but on routers they are processed in software, which can impact performance if the ACL has many entries. Best practice is to place the most specific and frequently matched entries at the top to reduce processing time. Misconfiguration can lead to security breaches or service outages. For example, if an ACL inadvertently permits all traffic from a malicious subnet, it can lead to data exfiltration. Conversely, blocking legitimate traffic can cause critical applications to fail, leading to downtime and user complaints. Always test ACL changes in a lab or during maintenance windows.