AWS WAF and Shield for SysOps

AWS WAF is a web application firewall that helps protect your web applications from common web exploits that could affect application availability, compromise security, or consume excessive resources. AWS Shield is a managed Distributed Denial of Service (DDoS) protection service that safeguards applications running on AWS. Both are tightly integrated and often work together.

AWS WAF operates at the edge (CloudFront) or on a regional Application Load Balancer (ALB), API Gateway, or AWS AppSync. When a client sends an HTTP(S) request, the request passes through the WAF before reaching the origin. The WAF evaluates the request against a set of rules defined in a Web Access Control List (web ACL). Each rule consists of a condition (what to inspect) and an action (allow, block, count, or CAPTCHA). Rules are evaluated in order; the first matching rule determines the action. If no rule matches, the default action (allow or block) is applied.

- Web ACL: A container for rules. You can associate a web ACL with one or more resources (ALB, CloudFront, API Gateway, AppSync). Each web ACL has a capacity unit (WCU) limit (default 1,500, adjustable). Rules consume WCUs based on complexity. - Rules: Define inspection criteria. Types include: - String match: Inspect headers, query strings, URI path, body, or cookies for a specific string or regex. - IP set: Match against a list of IP addresses or CIDR ranges. - Geo match: Match based on geographic origin (country). - Size constraint: Enforce minimum/maximum length of a request component. - SQL injection match: Detect SQL injection patterns. - Cross-site scripting (XSS) match: Detect XSS patterns. - Rate-based rule: Count requests from a source IP; if the count exceeds a threshold within a 5-minute window, the rule triggers (block or count). The rate is calculated every 30 seconds. - Rule groups: Preconfigured sets of rules. AWS provides managed rule groups (e.g., Core rule set, SQL database, Linux/Windows OS) and you can create your own. - Actions: - Allow: Request proceeds to the resource. - Block: Request is terminated with a 403 Forbidden response. - Count: Request is counted but allowed; used for testing. - CAPTCHA: Challenge the client to solve a puzzle. - Logging: Web ACL traffic can be logged to CloudWatch Logs, S3, or Kinesis Firehose. Logs include details like rule ID, action, and request metadata.

To create a web ACL via AWS CLI:

aws wafv2 create-web-acl \
    --name MyWebACL \
    --scope REGIONAL \
    --default-action Allow={} \
    --rules file://rules.json \
    --visibility-config SampledRequestsEnabled=true,CloudWatchMetricsEnabled=true,MetricName=MyWebACLMetric

Associate with an ALB:

aws wafv2 associate-web-acl \
    --web-acl-arn arn:aws:wafv2:us-east-1:123456789012:regional/webacl/MyWebACL/abc123 \
    --resource-arn arn:aws:elasticloadbalancing:us-east-1:123456789012:loadbalancer/app/my-alb/abc123

Verification: Check the web ACL's sampled requests via console or CLI:

aws wafv2 get-sampled-requests \
    --web-acl-arn arn:aws:wafv2:us-east-1:123456789012:regional/webacl/MyWebACL/abc123 \
    --rule-metric-name MyRuleMetric \
    --scope REGIONAL \
    --time-window Start=2023-01-01T00:00:00Z,End=2023-01-01T01:00:00Z \
    --max-items 100

AWS Shield Standard is automatically enabled for all AWS customers at no extra cost. It protects against common Layer 3/4 DDoS attacks (e.g., UDP floods, SYN floods) and provides always-on detection. AWS Shield Advanced is a paid service that provides enhanced protection against larger and more sophisticated DDoS attacks, including application layer (Layer 7) attacks. Shield Advanced integrates with AWS WAF to provide automated mitigations via rate-based rules and offers cost protection against scaling charges due to DDoS attacks.

Shield Advanced can automatically create and manage WAF rate-based rules for your CloudFront distributions or ALBs. When a DDoS attack is detected, Shield Advanced can apply a rate-based rule to block traffic from attacking IPs. You can also configure Shield Advanced to send notifications via CloudWatch alarms and AWS Lambda for custom responses.

Rate-based rules count requests from a source IP over a 5-minute sliding window. The count is updated every 30 seconds. If the count exceeds the threshold, the rule's action is applied to subsequent requests from that IP for the remainder of the window. After the window expires, the count resets.

A large e-commerce platform uses CloudFront with AWS WAF to protect against SQL injection and XSS attacks. They enable the AWS Managed Core Rule Set (CRS) and add a rate-based rule set to 5,000 requests per 5 minutes to mitigate bot scraping of product prices. They also use Shield Advanced for cost protection during flash sales when traffic spikes. In production, they observed false positives from the CRS blocking legitimate requests with unusual URL parameters (e.g., search queries containing 'select' or 'union'). They created custom rules to allow those patterns while still blocking malicious ones. Common misconfiguration: setting the rate-based rule threshold too low (e.g., 100 requests) caused legitimate bulk API calls to be blocked. Solution: use count mode initially to establish baseline traffic patterns.

A bank exposes REST APIs via API Gateway and uses WAF to enforce IP whitelisting for backend services. They use IP set rules to allow only known partner IPs and block all others. They also enable geo-match to restrict access to specific countries. To handle DDoS attacks, they subscribe to Shield Advanced, which automatically creates WAF rate-based rules when attack traffic exceeds thresholds. In production, they faced challenges with API Gateway's 30-second timeout for WAF evaluation; complex rules with many conditions caused latency. They optimized by minimizing rule count and using string match instead of regex where possible. Misconfiguration: forgetting to update IP sets when partners change IPs, leading to blocked legitimate traffic.

A SaaS provider uses ALB with WAF to protect tenant-specific microservices. They use WAF to block requests that contain malicious payloads in the body (e.g., JSON injection). They also use rate-based rules per tenant by leveraging the X-Tenant-Id header as a custom key (though WAF rate-based rules only support IP keys, they use a workaround with Lambda@Edge). They monitor WAF logs in CloudWatch Logs and trigger Lambda functions to auto-update IP sets when abuse is detected. Performance considerations: WAF adds ~10ms latency per request. Misconfiguration: not enabling CloudWatch metrics for each rule, making it hard to identify which rule is causing false positives.