This chapter covers Amazon Simple Queue Service (SQS) and Amazon Simple Notification Service (SNS), two core messaging services on AWS. For the DVA-C02 exam, messaging services appear in roughly 10-15% of questions, often in scenarios involving decoupling, asynchronous processing, and event-driven architectures. You must understand their mechanics, configurations, and how they interact with other services like Lambda, SQS Dead-Letter Queues, and SNS subscriptions. We'll dive deep into each service, including key parameters, defaults, and exam traps.
Jump to a section
Imagine a large company where departments need to send messages to each other. SQS is like a post office box system. Each department has a private PO box (queue). When Department A sends a message, it drops a letter into Department B's PO box. Department B checks its box periodically, picks up letters, processes them, and then discards them. If Department B is busy, letters just pile up in the box; the sender doesn't wait. The post office guarantees that each letter stays in the box until retrieved, and if retrieval fails, the letter goes back into the box for retry. Multiple mail clerks (consumers) can pull from the same PO box, but each letter is delivered to only one clerk. In contrast, SNS is like a company bulletin board. When someone posts a notice, it is pinned to the board, and multiple people (subscribers) can read it. The board doesn't track who has read it; it just broadcasts to everyone who is subscribed. If a subscriber misses the notice, it's gone (unless they have a durable subscription, like a personal copy). The key difference: SQS ensures each message is processed exactly once per queue (point-to-point), while SNS pushes messages to many subscribers simultaneously (publish-subscribe). On the exam, you must know when to use each: SQS for decoupling components where each message must be reliably processed, SNS for fan-out notifications to multiple endpoints.
Amazon SQS: Overview and Purpose
Amazon SQS is a fully managed message queuing service that enables decoupling of application components. It allows you to send, store, and receive messages between software components at any volume without losing messages or requiring other services to be always available. The exam focuses on two types of queues: Standard and FIFO (First-In-First-Out). Standard queues offer high throughput (nearly unlimited transactions per second) but guarantee at-least-once delivery, meaning duplicate messages are possible. FIFO queues guarantee exactly-once processing and preserve message order but are limited to 300 transactions per second (with batching up to 3,000). FIFO queues must have a .fifo suffix in their name.
How SQS Works Internally
When a producer sends a message to an SQS queue, the message is stored redundantly across multiple Availability Zones. The consumer polls the queue (short or long polling) to retrieve messages. Long polling (ReceiveMessageWaitTimeSeconds > 0) reduces empty responses and costs by waiting up to 20 seconds for a message to arrive. Short polling returns immediately even if the queue is empty. Once a consumer receives a message, it becomes invisible to other consumers for the duration of the Visibility Timeout (default 30 seconds, max 12 hours). The consumer must delete the message within that window. If not deleted, the message becomes visible again and may be reprocessed. This is how at-least-once delivery works. For FIFO queues, the consumer must also include a Receive Request Attempt ID to prevent duplicate processing.
Key SQS Parameters and Defaults
Visibility Timeout: 30 seconds default, adjustable per queue or per message. If a consumer fails to process and delete, the message reappears after timeout. Exam trap: Setting visibility timeout too low causes reprocessing; too high delays retries.
Message Retention Period: Default 4 days, max 14 days. Messages not deleted after retention expire are automatically deleted.
Maximum Message Size: 256 KB (including body and metadata). For larger messages, use S3 with SQS Extended Client Library (exam expects this pattern).
Dead-Letter Queue (DLQ): A separate queue (Standard or FIFO) that receives messages after a configurable number of receive attempts (maxRedriveReceiveCount, 1-1000). DLQs are essential for handling poison pills (messages that repeatedly fail processing).
Delay Queue: Delays initial message delivery by up to 15 minutes (DelaySeconds parameter). For individual messages, use message timers.
Request Pricing: Pay per request (each 64KB chunk counts as one request). Long polling reduces costs by reducing empty responses.
SQS Polling and Consumer Patterns
Consumers can poll using the AWS SDK, CLI, or Lambda (Lambda event source mapping polls internally). The exam tests understanding of short vs. long polling. Short polling samples a subset of servers; it may return empty even if messages exist. Long polls all servers, returning as soon as a message is available or until timeout. For high-traffic queues, use long polling to reduce cost and latency. For FIFO queues, consumers must use a unique ReceiveRequestAttemptId to avoid duplicate processing.
Amazon SNS: Overview and Purpose
SNS is a pub/sub messaging service that enables you to send messages to a large number of subscribers (endpoints) via topics. Subscribers can be SQS queues, Lambda functions, HTTP/HTTPS endpoints, email, SMS, and platform application endpoints (mobile push). The exam tests SNS mainly in fan-out patterns, where one SNS topic sends to multiple SQS queues. SNS also supports message filtering (by attributes) so subscribers receive only relevant messages.
How SNS Works Internally
When a publisher sends a message to an SNS topic, SNS immediately pushes the message to all subscribers. Delivery is asynchronous and retried according to a retry policy (for HTTP/S endpoints). SNS uses a message durability model: for SQS and Lambda subscribers, the message is durable if the subscriber accepts it. For HTTP/S, SNS retries up to 3 times by default, then either discards or sends to a DLQ (if configured). SNS does NOT store messages persistently; if no subscriber is available, the message is lost. This is a critical distinction from SQS.
Key SNS Parameters and Defaults
Topic Types: Standard (high throughput, no ordering) and FIFO (ordered, exactly-once, requires SQS FIFO subscribers). FIFO topics must have .fifo suffix.
Message Durability: SNS does not store messages. For durable delivery, subscribe an SQS queue to the topic (fan-out pattern).
Message Filtering: Subscribers can set a filter policy based on message attributes (not body). Filter policies are JSON strings.
Delivery Retry Policy: For HTTP/S endpoints, you can set NumberOfRetries (default 3), RetryPolicy (exponential backoff).
Dead-Letter Queue: For SNS, you can attach a DLQ to a subscription (not the topic) to capture failed deliveries.
Message Size: Same as SQS, 256 KB max.
SNS and SQS Integration (Fan-out Pattern)
The classic fan-out pattern: one SNS topic publishes to multiple SQS queues, each serving a different consumer. This allows parallel processing and decoupling. Exam scenario: A new order event triggers an SNS topic, which fans out to an SQS queue for inventory, another for shipping, and another for analytics. Each SQS queue can be polled independently. If a consumer fails, messages pile up in its queue without affecting others.
Interaction with Lambda
Lambda can poll SQS queues (event source mapping) or be subscribed to SNS topics. For SQS, Lambda reads messages in batches (configurable batch size, up to 10,000). Lambda's event source mapping supports partial batch responses (reporting individual item failures) for Standard queues. For SNS, Lambda receives the SNS event directly; the message is passed as a JSON payload. The exam tests that SNS triggers Lambda synchronously (Lambda function must be subscribed to the topic).
Security and Encryption
Both SQS and SNS support server-side encryption (SSE) using KMS. You can also enforce encryption at rest and in transit using HTTPS endpoints. Access control is via IAM policies (for queues/topics) and queue/topic policies (resource-based). The exam tests that SQS and SNS support VPC endpoints (interface endpoints) to keep traffic within AWS network.
Monitoring and Metrics
CloudWatch metrics for SQS: ApproximateNumberOfMessagesVisible, ApproximateAgeOfOldestMessage, NumberOfMessagesSent, etc. For SNS: NumberOfMessagesPublished, NumberOfNotificationsDelivered, NumberOfNotificationsFailed. DLQ metrics help detect processing failures. Exam tip: If a message is not being processed, check the DLQ for poison pills.
CLI Examples
# Create a Standard queue
aws sqs create-queue --queue-name MyQueue
# Send a message
aws sqs send-message --queue-url https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue --message-body "Hello"
# Receive a message
aws sqs receive-message --queue-url https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue --wait-time-seconds 10
# Create an SNS topic
aws sns create-topic --name MyTopic
# Subscribe an SQS queue to the topic
aws sns subscribe --topic-arn arn:aws:sns:us-east-1:123456789012:MyTopic --protocol sqs --notification-endpoint arn:aws:sqs:us-east-1:123456789012:MyQueue
# Publish to topic
aws sns publish --topic-arn arn:aws:sns:us-east-1:123456789012:MyTopic --message "Hello"Producer sends message to SQS
The producer application calls the SendMessage API with the queue URL, message body (up to 256KB), optional delay seconds, message attributes, and for FIFO queues, a message group ID and deduplication ID. SQS stores the message redundantly across multiple AZs and returns a message ID. The producer does not wait for a consumer; it is fully decoupled.
Consumer polls SQS queue
The consumer calls ReceiveMessage API. For long polling, set WaitTimeSeconds (0-20). SQS checks all servers for available messages. If found, it returns up to MaxNumberOfMessages (1-10) and makes them invisible for the VisibilityTimeout period. For FIFO queues, the consumer must provide a ReceiveRequestAttemptId to prevent duplicate processing.
Consumer processes and deletes message
The consumer processes the message within the visibility timeout. After successful processing, it calls DeleteMessage with the receipt handle. If the consumer fails or times out, the message becomes visible again after the visibility timeout expires. The message can be reprocessed up to the maxReceiveCount before being sent to the DLQ.
Publisher sends message to SNS topic
The publisher calls Publish API with topic ARN, message body, optional subject and message attributes. SNS immediately attempts to deliver the message to all subscribers. For SQS subscribers, SNS writes the message to the queue. For Lambda, it invokes the function. For HTTP/S, it sends an HTTP POST. The publisher does not wait for delivery confirmation.
SNS retries failed deliveries
For HTTP/S endpoints, if the endpoint returns an error or times out, SNS retries according to the subscription's retry policy (default 3 retries with exponential backoff). After exhausting retries, the message can be sent to a configured DLQ. For SQS and Lambda, if the subscriber is unavailable, SNS retries internally. For email, delivery is best-effort.
Enterprise Scenario 1: E-commerce Order Processing
An e-commerce platform uses SNS to fan out order events. When a customer places an order, the order service publishes an event to an SNS topic. Subscribers include: an SQS queue for inventory updates, another for shipping label generation, and a Lambda function for order confirmation emails. This decouples the order service from downstream processing. The inventory team can scale their workers independently. If the shipping service is down, messages accumulate in its queue without affecting order capture. Misconfiguration: If the SNS topic's message retention is not considered (SNS does not persist), a subscriber that is added later will not receive past events. The solution is to use SQS FIFO queues for ordering and deduplication, and ensure DLQs are configured to catch failures.
Enterprise Scenario 2: Microservices Decoupling
A financial services company uses SQS to decouple transaction processing. A front-end service sends transaction data to an SQS queue. Multiple backend workers poll the queue for processing. Each worker processes a transaction and deletes it. If a worker crashes mid-processing, the message reappears after visibility timeout and is picked up by another worker. This ensures no transactions are lost. The team uses CloudWatch alarms on ApproximateAgeOfOldestMessage to detect processing delays. They set a DLQ to capture messages that fail after 3 retries. Common pitfall: Setting visibility timeout too low (e.g., 5 seconds) causes excessive reprocessing and duplicates; too high (e.g., 1 hour) delays failure recovery.
Enterprise Scenario 3: Mobile Push Notifications
A social media app uses SNS to send push notifications to millions of users. The app server publishes a message to an SNS topic for each notification type (like, comment, follow). SNS delivers to platform application endpoints (APNs for iOS, FCM for Android). The team uses message filtering to send only relevant notifications to each device. They monitor delivery failures and use DLQs to capture undelivered messages. Scaling: SNS can handle millions of messages per second. However, if the app server publishes too many messages, they may hit throttling limits. They use batching (up to 10 messages per Publish call) to improve throughput.
What DVA-C02 Tests on SQS and SNS
This topic falls under Domain 1: Development (Objective 1.5: Implement event-driven architectures). Expect 2-4 questions on SQS and SNS combined. The exam focuses on:
Differences between Standard and FIFO queues (ordering, exactly-once, throughput limits)
Visibility timeout mechanics and its effect on message processing
Dead-Letter Queue configuration and use cases
Long polling vs short polling
SNS fan-out pattern to multiple SQS queues
SNS message filtering based on attributes
SQS Extended Client Library for messages >256KB
FIFO topics and their requirement for FIFO SQS subscribers
Common Wrong Answers and Why
"SNS stores messages for up to 14 days." Wrong: SNS does not store messages. SQS stores messages. Candidates confuse the two.
"SQS guarantees exactly-once delivery for Standard queues." Wrong: Standard queues guarantee at-least-once. FIFO queues guarantee exactly-once.
"Visibility timeout can be set to 0." Wrong: Minimum is 1 second. Setting it too low causes duplicates.
"SNS can send messages directly to Lambda without a subscription." Wrong: Lambda must be subscribed to the SNS topic. The exam tests that SNS invokes Lambda synchronously.
Specific Numbers and Terms
FIFO throughput: 300 tps (without batching), 3000 tps (with batching of up to 10 messages per batch).
SQS message retention: 4 days default, 14 days max.
Visibility timeout: 30 seconds default, 1 second to 12 hours.
Long polling wait time: 0-20 seconds.
Max message size: 256 KB.
DLQ maxReceiveCount: 1-1000.
SNS topic name: must be unique within a region.
FIFO queue name: must end with .fifo.
Edge Cases and Exceptions
Empty response with short polling: Short polling samples a subset of servers; it may return empty even if messages exist. Long polling queries all servers.
Message timers vs delay queues: Delay queues apply to all messages; message timers apply per message and override queue delay.
FIFO queues with Lambda: Lambda processes messages in order per message group ID. If a message fails, all subsequent messages in that group are blocked until the first succeeds or reaches DLQ.
SNS message filtering: Only works on message attributes, not body. Filter policy is a JSON object with attribute name, value, and match type (exact, prefix, etc.).
How to Eliminate Wrong Answers
If the question mentions ordering and exactly-once, look for FIFO. If it mentions high throughput and duplicates acceptable, Standard.
If the scenario requires decoupling with persistent storage, choose SQS. If it requires broadcasting to multiple consumers, choose SNS.
If a message is too large, look for SQS Extended Client Library (stores payload in S3).
If a consumer fails to process, look for DLQ configuration.
SQS Standard queues: at-least-once, high throughput, unordered. FIFO queues: exactly-once, ordered, 300 TPS (3000 with batching).
SQS visibility timeout: default 30 seconds, range 1 second to 12 hours. Set based on expected processing time.
SQS long polling: set ReceiveMessageWaitTimeSeconds > 0 (max 20) to reduce empty responses and cost.
SNS does not store messages; use SQS subscription for durable delivery.
SNS message filtering: based on message attributes, not body. Use JSON filter policy.
DLQ for SQS: configure redrive policy with maxReceiveCount (1-1000). For SNS: attach DLQ to subscription.
Messages >256KB: use SQS Extended Client Library (stores payload in S3).
FIFO topics require FIFO SQS subscribers. Both must have .fifo suffix.
Lambda can poll SQS (event source mapping) or be triggered by SNS (subscription).
Both SQS and SNS support SSE-KMS and VPC endpoints.
These come up on the exam all the time. Here's how to tell them apart.
SQS (Standard Queue)
High throughput: nearly unlimited TPS
At-least-once delivery (duplicates possible)
No ordering guarantee
Name does not require suffix
Supports Lambda partial batch response
SQS (FIFO Queue)
Limited to 300 TPS (3000 with batching)
Exactly-once processing (no duplicates)
Guarantees FIFO ordering within message group
Name must end with .fifo
Does not support partial batch response
SQS
Pull-based: consumers poll for messages
Persistent storage (retention up to 14 days)
Point-to-point: each message delivered to one consumer
Supports delay queues and message timers
Can be used as DLQ for other services
SNS
Push-based: pushes messages to subscribers
No persistent storage; messages lost if not delivered
Pub/sub: one message delivered to many subscribers
Supports message filtering by attributes
Can have DLQ attached to subscriptions
Mistake
SNS stores messages until a subscriber consumes them.
Correct
SNS does not store messages. It pushes messages to subscribers immediately and does not persist them. For durable storage, subscribe an SQS queue to the SNS topic.
Mistake
SQS Standard queues guarantee exactly-once delivery.
Correct
Standard queues guarantee at-least-once delivery. Duplicates can occur. FIFO queues guarantee exactly-once processing.
Mistake
Visibility timeout can be set to 0 seconds.
Correct
Minimum visibility timeout is 1 second. Setting it too low causes excessive reprocessing and duplicates.
Mistake
SNS can deliver messages to Lambda without a subscription.
Correct
Lambda must be subscribed to the SNS topic. SNS invokes the Lambda function synchronously.
Mistake
SQS messages can be up to 2 MB in size.
Correct
The maximum message size is 256 KB. For larger messages, use the SQS Extended Client Library which stores the payload in S3.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
Standard queues offer high throughput (nearly unlimited TPS) but only guarantee at-least-once delivery, meaning duplicates are possible and messages are not ordered. FIFO queues guarantee exactly-once processing and preserve message order within a message group, but are limited to 300 TPS (3000 with batching). FIFO queue names must end with '.fifo'. Use Standard when throughput is critical and duplicates are acceptable; use FIFO when ordering and deduplication are required.
When a consumer receives a message, it becomes invisible to other consumers for the duration of the visibility timeout (default 30 seconds, configurable 1 second to 12 hours). The consumer must delete the message within that window. If not deleted, the message becomes visible again and can be reprocessed. This ensures at-least-once delivery. Setting the timeout too low causes duplicate processing; too high delays failure recovery.
Use SNS when you need to broadcast a message to multiple consumers (pub/sub pattern). For example, an order event that must trigger inventory updates, shipping, and notifications. Use SQS when you need decoupled, point-to-point communication where each message is processed by a single consumer. SNS does not store messages, so for durable delivery, subscribe an SQS queue to the SNS topic (fan-out).
A DLQ is a separate SQS queue (Standard or FIFO) that receives messages that failed processing after a maximum number of attempts. For SQS, configure a redrive policy with maxReceiveCount (1-1000). For SNS, attach a DLQ to a subscription. DLQs help isolate problematic messages (poison pills) for analysis. Ensure the DLQ is of the same type (Standard/FIFO) as the source queue.
Use the SQS Extended Client Library. It stores the message payload in an S3 bucket and sends a reference (S3 key) in the SQS message. The consumer retrieves the payload from S3. This pattern is transparent to the application. The exam tests this as the solution for large messages.
Short polling (ReceiveMessageWaitTimeSeconds=0) returns immediately, even if the queue is empty. It samples only a subset of servers, so it may miss messages. Long polling (WaitTimeSeconds 1-20) waits for a message to arrive, queries all servers, and reduces empty responses and cost. Long polling is recommended for most use cases.
Yes, you can subscribe a Lambda function to an SNS topic. When a message is published, SNS invokes the Lambda function synchronously with the message payload. The Lambda function must have a resource-based policy allowing SNS to invoke it. Note that Lambda is not a durable endpoint; if the function fails, SNS retries according to its retry policy.
You've just covered SQS and SNS for Developers — now see how well it sticks with free DVA-C02 practice questions. Full explanations included, no account needed.
Done with this chapter?