This chapter covers Amazon SNS message filtering and the fan-out pattern, a core architectural pattern for decoupled, event-driven systems on AWS. For the SAA-C03 exam, understanding SNS filtering and fan-out is essential because it appears in approximately 10% of questions related to application integration and decoupling (Objective 3.2). You will learn how SNS topics distribute messages to multiple subscribers, how filter policies selectively deliver messages, and how to combine SNS with SQS, Lambda, and other services to build scalable, fault-tolerant solutions. Mastering these concepts will help you answer scenario-based questions about decoupling microservices, broadcasting events, and implementing message routing.
Jump to a section
Imagine a large postal sorting hub that receives a single shipment of identical letters from a company. Each letter is addressed to 'Customer Service Department' generically. Inside the hub, a sorting machine reads the generic address and automatically makes photocopies of each letter. The machine then applies different routing rules per batch: one batch gets an additional 'Urgent' stamp and is sent to the email processing line, another batch gets a 'Confidential' sticker and goes to a secure fax line, and a third batch is left as-is and sent to standard postal mail. The key is that the hub does not decide which letters to forward—it forwards every letter to every configured destination, but each destination can apply its own filter (e.g., only accept letters with 'Urgent' stamp) before processing. If a destination's line is full, the hub holds the letter in a dedicated queue for that destination until it can accept it. The hub never modifies the original letter; it only makes copies and optionally attaches metadata (like stamps). This mirrors SNS: the topic receives a message, creates copies for each subscription, and sends them to their respective endpoints (SQS, Lambda, HTTP, etc.). Each subscription can have a filter policy that acts like the 'Urgent' stamp check—only messages matching the policy are delivered. If an endpoint is unavailable, SNS retries or sends to a dead-letter queue, analogous to the hub holding letters for a busy line.
What is SNS Message Filtering and the Fan-Out Pattern?
Amazon Simple Notification Service (SNS) is a fully managed pub/sub messaging service. A topic is a logical access point that acts as a communication channel. Publishers send messages to a topic, and SNS immediately pushes those messages to all subscribers of that topic. The fan-out pattern refers to the architecture where a single SNS topic delivers messages to multiple subscriber endpoints (e.g., SQS queues, Lambda functions, HTTP/HTTPS endpoints, email, SMS, platform application endpoints) simultaneously, enabling parallel processing and decoupling.
Message filtering adds a crucial capability: without filtering, every subscriber receives every message published to the topic. With filter policies, subscribers can define rules based on message attributes (not the message body) to receive only a subset of messages. This reduces unnecessary processing, network traffic, and costs.
How It Works Internally
When a message is published to an SNS topic, the following occurs:
1. Ingestion: The publisher sends a request to the SNS API (Publish action) with the topic ARN, message body, and optionally message attributes (key-value pairs with types: String, String.Array, Number, Binary). 2. Replication: SNS replicates the message across multiple Availability Zones for durability. 3. Subscription Evaluation: For each subscription, SNS evaluates the filter policy (if any) against the message attributes. The filter policy is a JSON object that specifies matching criteria (e.g., { "event": ["order_placed"] }). Only messages whose attributes match the policy are delivered to that subscriber. 4. Delivery: If the filter matches, SNS attempts to deliver the message to the endpoint. Delivery semantics vary by protocol: - SQS: SNS writes the message directly into the subscribed SQS queue. The SQS queue must have a resource-based policy allowing SNS to send messages. - Lambda: SNS invokes the Lambda function asynchronously with the message payload. - HTTP/HTTPS: SNS sends an HTTP POST request to the endpoint. The endpoint must respond with a 200 OK to confirm receipt; otherwise, SNS retries (with exponential backoff) or sends to a dead-letter queue (DLQ) if configured. - Email/Email-JSON: SNS sends an email via Amazon SES. - SMS: SNS sends a text message via SMS providers. - Platform Application: SNS sends a push notification to mobile devices via Apple Push Notification Service (APNS), Google Firebase Cloud Messaging (FCM), etc. 5. Retry and DLQ: If delivery fails, SNS retries based on the subscription's retry policy (default: 3 retries, then message discarded unless a DLQ is configured). A dead-letter queue (SQS) can capture undeliverable messages for analysis.
Key Components, Values, Defaults, and Timers
Message Attributes: Up to 10 attributes per message. Each attribute has a name (alphanumeric, up to 256 bytes), type (String, String.Array, Number, Binary), and value (max 256 KB total message size, including body and attributes).
Filter Policy: Defined per subscription as a JSON object. Example: {"event": ["order_placed", "order_shipped"]}. Supports exact matching, prefix matching, numeric matching (>, >=, <, <=, ==, !=), and anything-but matching. Attribute keys must match exactly; values are matched against the policy list (any match triggers delivery).
Subscription Confirmation: For HTTP/HTTPS and email subscriptions, the subscriber must confirm the subscription before receiving messages. SNS sends a confirmation request to the endpoint; the endpoint must respond with a GET request containing the token. For SQS, Lambda, and other AWS services, confirmation is automatic.
Message Size: Maximum 256 KB per message (including body and attributes).
Retry Policy: Default: 3 retries with exponential backoff (immediate, then ~2 seconds, then ~4 seconds). Customizable via AWS Support or API (not directly in console).
Dead-Letter Queue: Must be an SQS queue (standard or FIFO) in the same AWS account and region. DLQ must have a resource policy allowing SNS to send messages.
Delivery Status Logging: Can be enabled to log delivery failures to CloudWatch Logs.
FIFO Topics: Support exactly-once delivery and message ordering. Filter policies are supported. Subscribers must be SQS FIFO queues.
Message Deduplication: For FIFO topics, deduplication ID is required to prevent duplicates.
Configuration and Verification
Creating a Filter Policy (AWS CLI):
aws sns subscribe \
--topic-arn arn:aws:sns:us-east-1:123456789012:MyTopic \
--protocol sqs \
--notification-endpoint arn:aws:sqs:us-east-1:123456789012:MyQueue \
--attributes '{"FilterPolicy": "{\"event\": [\"order_placed\"]}"}'Verifying Filter Policy:
aws sns get-subscription-attributes --subscription-arn arn:aws:sns:us-east-1:123456789012:MyTopic:abc123This returns the subscription attributes, including FilterPolicy.
Publishing a message with attributes:
aws sns publish \
--topic-arn arn:aws:sns:us-east-1:123456789012:MyTopic \
--message "Order placed" \
--message-attributes '{"event": {"DataType": "String", "StringValue": "order_placed"}}'Interaction with Related Technologies
SNS + SQS (Fan-Out): The most common pattern. Each subscriber is an SQS queue. This allows multiple consumer groups to process messages independently, with buffering and retries at the queue level. The SQS queue must have a policy allowing SNS to send messages.
SNS + Lambda: Directly invokes a Lambda function. Useful for lightweight processing or triggering workflows. Lambda can also be used to filter messages before further processing.
SNS + Kinesis Data Firehose: Can stream messages to destinations like S3, Redshift, or Elasticsearch.
SNS + CloudWatch: Can publish CloudWatch alarms to an SNS topic for notification.
SNS + AWS Chatbot: Delivers notifications to Slack or Microsoft Teams channels.
SNS + EventBridge: SNS can be a target for EventBridge rules, enabling complex event routing.
Performance and Scaling
SNS automatically scales to handle high throughput. There are no limits on the number of subscribers per topic (though AWS API has a default limit of 12,500,000 subscriptions per topic, soft limit can be increased). Each message is delivered to each subscriber independently; if a subscriber is slow, it does not affect other subscribers because delivery is parallel. For high-volume scenarios, use SQS queues as subscribers to decouple consumers and provide buffering.
Create SNS Topic
First, create an SNS topic (standard or FIFO). The topic ARN is the unique identifier. For standard topics, messages are delivered at least once; for FIFO topics, exactly once and in order. FIFO topics require .fifo suffix in the name. The topic serves as the central hub for the fan-out pattern.
Create Subscriber Endpoints
Create the endpoints that will receive messages. Common endpoints include SQS queues (standard or FIFO), Lambda functions, HTTP/HTTPS endpoints, email addresses, and mobile push endpoints. For SQS, note the queue ARN and ensure the queue policy allows SNS to send messages. For Lambda, the function must have a resource-based policy allowing SNS invocation.
Subscribe Endpoints to Topic
Subscribe each endpoint to the SNS topic using the AWS Console, CLI, or SDK. For each subscription, you can optionally attach a filter policy (JSON) that specifies which messages to deliver. For HTTP/HTTPS, the subscriber must confirm the subscription by responding to a confirmation request. For AWS services like SQS and Lambda, confirmation is automatic.
Configure Filter Policies
Define filter policies per subscription as a JSON object. The policy is evaluated against message attributes (not body). Example: {"event": ["order_placed"]}. Only messages with an attribute 'event' whose value is exactly 'order_placed' will be delivered. Filter policies can include multiple keys (AND logic within keys, OR logic across values in a list). Supports prefix matching ({"event": [{"prefix": "order_"}]}), numeric matching ({"price": [{"numeric": [">=", 100, "<", 200]}]}), and anything-but ({"event": [{"anything-but": "order_cancelled"}]}).
Publish Message with Attributes
Publishers send messages to the topic using the Publish API. Include message attributes that the filter policies will evaluate. Attributes are key-value pairs with a data type (String, String.Array, Number, Binary). Up to 10 attributes per message. The message body can be any text (JSON, XML, plain text) up to 256 KB total. SNS replicates the message and evaluates each subscription's filter policy.
Monitor and Handle Failures
Enable delivery status logging to CloudWatch to monitor successful and failed deliveries. Configure dead-letter queues (SQS) for subscriptions to capture messages that cannot be delivered after retries. For HTTP endpoints, ensure they return 200 OK within a timeout (default 15 seconds). Use CloudWatch alarms to alert on high failure rates.
Enterprise Scenario 1: E-commerce Order Processing
A large e-commerce platform uses SNS to decouple order processing. When a customer places an order, the order service publishes a message to an SNS topic with attributes like event: order_placed, customer_tier: gold, and amount: 150.00. Multiple downstream services subscribe to the topic with filter policies:
Inventory Service: Subscribes with filter {"event": ["order_placed"]} to update stock levels.
Shipping Service: Subscribes with filter {"event": ["order_shipped"]}.
Notification Service: Subscribes with filter {"event": ["order_placed", "order_shipped", "order_delivered"], "customer_tier": ["gold", "platinum"]} to send high-priority notifications.
Analytics Service: Subscribes with no filter to capture all events for data warehousing.
Each subscriber uses an SQS queue to buffer messages and allow independent scaling. The system processes over 10,000 orders per minute. Without filtering, the notification service would receive every order, including low-tier customers, causing unnecessary processing. The dead-letter queue catches messages from misconfigured services.
Enterprise Scenario 2: Multi-Region Replication with SNS
A global SaaS provider replicates configuration changes across regions. In the primary region (us-east-1), an SNS topic receives configuration updates. Subscribers include SQS queues in separate AWS accounts and regions, each with a filter policy to receive only changes relevant to that region (e.g., {"region": ["eu-west-1"]}). The SQS queues are cross-region, and Lambda functions poll these queues to apply changes locally. This pattern ensures that only relevant messages cross regions, reducing inter-region data transfer costs. The system handles 500 updates per second. A common misconfiguration is forgetting to update the SQS queue policy to allow SNS from another account/region.
Performance and Misconfiguration Pitfalls
Too many subscribers: While SNS scales, each subscriber adds latency because SNS must evaluate filter policies. For thousands of subscribers, consider using a fan-out to SQS first, then have consumers poll the queue.
Filter policy too complex: Filter policies are evaluated per message; complex policies (many keys, nested conditions) can increase latency. Keep policies simple.
Message attributes not sent: If publishers forget to include attributes, all messages are delivered to all subscribers (since no filter matches, the default is to deliver all). This is a common cause of unexpected behavior.
Dead-letter queue misconfiguration: If the DLQ is not properly configured, undeliverable messages are lost. Always set up DLQs for critical subscriptions.
Subscription confirmation timeout: For HTTP endpoints, if the endpoint does not confirm within 3 days, the subscription is deleted. Ensure endpoints respond promptly.
What the SAA-C03 Exam Tests (Objective 3.2)
The exam focuses on designing decoupled architectures using SNS and SQS. Key areas: - Fan-out pattern: Recognize when to use SNS to broadcast messages to multiple consumers. - Filter policies: Understand that filtering is based on message attributes, not the body. The exam may present a scenario requiring selective delivery. - Combining SNS with SQS: Know that SQS must have a resource-based policy allowing SNS to send messages. The queue can be standard or FIFO, but FIFO topics require FIFO queues. - Dead-letter queues: Know that DLQs are configured at the subscription level (for SNS) or queue level (for SQS). - Retry behavior: Default 3 retries, then message discarded unless DLQ is set. - Message size limit: 256 KB total. - FIFO topics: Support ordering and exactly-once delivery; require deduplication ID and group ID for ordering.
Common Wrong Answers and Why
"Use SQS with a single queue and filter consumers": Candidates often suggest SQS alone because they are more familiar with it. However, SQS does not support server-side filtering; consumers must poll and filter themselves, which is inefficient. The correct answer is SNS with filter policies.
"Filter policies can filter on message body": Many candidates think filtering is based on the body. The exam explicitly tests that filtering uses message attributes. A question may describe a scenario where the body contains a JSON key; the correct answer is to add that key as an attribute.
"SNS FIFO topics can have any subscriber type": FIFO topics only support SQS FIFO queues as subscribers. HTTP, Lambda, etc., are not supported. Candidates often forget this.
"Dead-letter queues are configured on the topic": DLQs are configured per subscription, not at the topic level. Each subscription can have its own DLQ.
Specific Numbers and Terms
256 KB: Maximum message size.
10: Maximum number of message attributes.
3: Default retry count.
.fifo: Suffix required for FIFO topics.
FilterPolicy: Attribute name for filter policy in subscription attributes.
RedrivePolicy: Attribute for dead-letter queue configuration.
Edge Cases and Exceptions
If a filter policy is empty or not set, all messages are delivered.
If a filter policy is malformed, the subscription becomes "Pending Confirmation" and messages are not delivered.
FIFO topic subscribers must be FIFO queues; standard queues are not allowed.
Cross-account subscriptions: The subscriber's endpoint must have a policy allowing the SNS topic's account to send.
Message attributes with binary data are base64-encoded.
How to Eliminate Wrong Answers
If the question mentions "selective delivery" or "only send certain messages," look for SNS with filter policies.
If the question mentions "order" or "exactly-once," consider FIFO topics and FIFO queues.
If the question mentions "buffering" or "decouple processing," think SQS as subscriber.
If the question mentions "broadcast" or "multiple consumers," think SNS fan-out.
SNS fan-out pattern delivers a single message to multiple subscribers simultaneously, enabling decoupled architectures.
Filter policies are JSON objects attached to subscriptions that match on message attributes (not body).
FIFO topics require .fifo suffix and only support SQS FIFO queue subscribers.
Default retry policy for failed deliveries is 3 attempts with exponential backoff; configure a dead-letter queue to capture undeliverable messages.
Maximum message size is 256 KB (including attributes).
Message attributes support types: String, String.Array, Number, Binary; up to 10 attributes per message.
Subscription confirmation is required for HTTP/HTTPS and email endpoints; AWS service endpoints are auto-confirmed.
Dead-letter queues must be SQS queues in the same account and region as the SNS subscription.
SNS scales automatically; no limit on number of subscribers per topic (default soft limit 12.5 million).
For cross-account subscriptions, the subscriber's endpoint must have a resource policy allowing the SNS topic's account to send.
These come up on the exam all the time. Here's how to tell them apart.
SNS with Filter Policies
Server-side filtering reduces network traffic and consumer load.
Filter policies are defined per subscription, evaluated by SNS.
Supports attribute-based filtering (exact, prefix, numeric, anything-but).
Ideal for fan-out to multiple consumers with different interests.
No need for consumers to poll and filter; SNS pushes only matching messages.
SQS with Consumer-Side Filtering
All messages are delivered to the queue; consumers must poll and filter themselves.
Filtering logic is implemented in consumer code, increasing complexity.
Can filter on any part of the message (body, attributes) after retrieval.
Suitable for single consumer or when all consumers need all messages.
Consumers must manage their own filtering logic, potentially wasting compute.
Mistake
SNS filter policies can filter based on the message body.
Correct
Filter policies only evaluate message attributes, not the body. To filter on body content, you must extract relevant data into attributes before publishing.
Mistake
SNS FIFO topics support all subscriber types (HTTP, Lambda, etc.).
Correct
FIFO topics only support SQS FIFO queues as subscribers. Other protocols like HTTP, Lambda, email, or SMS are not supported for FIFO topics.
Mistake
If no filter policy is set, no messages are delivered.
Correct
Without a filter policy, all messages are delivered to the subscriber. An empty filter policy acts as a wildcard, delivering everything.
Mistake
Dead-letter queues can be any type of queue (SQS, SNS, etc.).
Correct
Dead-letter queues for SNS subscriptions must be SQS queues (standard or FIFO) in the same AWS account and region. SNS cannot use another SNS topic as a DLQ.
Mistake
Message filtering reduces the number of messages published to the topic.
Correct
Filtering happens after publication, at the subscription level. The topic still receives all messages; filtering only affects which subscribers receive them. You still pay for publishing all messages.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
No, SNS filter policies only work on message attributes, not the message body. To filter on body content, you must extract the relevant data and include it as an attribute when publishing the message. Alternatively, you can use EventBridge with content-based filtering on the event payload, but that's a different service.
SNS standard topics offer at-least-once delivery and high throughput, but messages may be duplicated or out of order. FIFO topics provide exactly-once delivery and strict message ordering within a message group. FIFO topics require a .fifo suffix in the name and only support SQS FIFO queues as subscribers. They also require a message deduplication ID or content-based deduplication.
You can configure a dead-letter queue (DLQ) for an SNS subscription by setting the RedrivePolicy attribute. The DLQ must be an SQS queue (standard or FIFO) in the same AWS account and region. Use the AWS CLI: aws sns set-subscription-attributes --subscription-arn <arn> --attribute-name RedrivePolicy --attribute-value '{"deadLetterTargetArn":"arn:aws:sqs:us-east-1:123456789012:MyDLQ"}'.
Yes, but you need to configure cross-account access. The Lambda function in the other account must have a resource-based policy that allows the SNS topic's account to invoke it. Additionally, the SNS subscription can be created from the topic owner's account by specifying the Lambda ARN from the other account.
If you attach an invalid filter policy (e.g., malformed JSON), the subscription will remain in 'Pending Confirmation' state for HTTP/HTTPS endpoints, or may fail to create for other protocols. The subscription will not receive any messages until the policy is corrected. Always validate your filter policy JSON before applying.
There is no limit on the number of filter policies per topic, but each subscription can have only one filter policy. The total number of subscriptions per topic is limited by a soft limit (default 12.5 million, can be increased). Each filter policy can contain multiple conditions.
No, you pay for every message published to the topic, regardless of how many subscribers receive it after filtering. You also pay for delivery to each subscriber. Filtering does not reduce the publish cost, but it can reduce downstream processing costs and network traffic.
You've just covered SNS Message Filtering and Fan-Out Pattern — now see how well it sticks with free SAA-C03 practice questions. Full explanations included, no account needed.
Done with this chapter?