This chapter covers Amazon SQS FIFO queues, focusing on strict ordering and exactly-once processing. FIFO queues are critical for applications that require messages to be processed in the exact order they are sent and without duplicates. On the DVA-C02 exam, approximately 5-8% of questions touch SQS, with FIFO-related topics appearing in about 2-3% of total questions. You will be tested on the mechanics of message ordering, deduplication, throughput limits, and how FIFO queues differ from standard queues.
Jump to a section
Imagine a factory assembly line for custom bicycles. Each bicycle order comes with a unique order number and a customer name. The factory floor has a strict rule: orders must be processed in the exact sequence they arrive, and no duplicate orders for the same customer can be processed at the same time. The line starts with a single conveyor belt (the FIFO queue). Each order is placed on the belt with a unique ID (the message deduplication ID). The first worker takes the first order off the belt, works on it, and when done, sends a signal back to the belt operator (the consumer deletes the message). If an order has the same customer name and order number (duplicate), the belt operator sees the unique ID and refuses to place it on the belt again during a 5-minute window (the deduplication interval). The belt operator also ensures that even if two orders arrive at the exact same millisecond, they are placed in a precise order based on their group ID (message group ID) — orders for the same customer go into the same group, and within that group, order is strictly preserved. If the first worker's machine jams, the order stays on the belt until it is successfully processed or a timeout expires (the visibility timeout). This system guarantees that every customer's orders are processed in the right order and no order is ever duplicated, even if the same order is accidentally submitted twice.
What is an SQS FIFO Queue?
Amazon Simple Queue Service (SQS) FIFO (First-In-First-Out) queues are designed to guarantee that messages are processed exactly once, in the exact order they are sent. Unlike standard SQS queues which offer best-effort ordering and at-least-once delivery, FIFO queues provide strict ordering and exactly-once processing within a message group. The name "FIFO" refers to the order of delivery: the first message sent is the first message delivered to a consumer, provided the consumer processes and deletes messages in order.
Why Does FIFO Exist?
Standard SQS queues are highly scalable and provide at-least-once delivery, but they do not guarantee order. For many applications—such as financial transactions, order processing, inventory updates, and event sourcing—the order of messages matters. Processing a "debit" before a "credit" could cause incorrect balances. FIFO queues solve this by enforcing strict ordering. Additionally, standard queues may deliver the same message multiple times (at-least-once), requiring idempotent consumers. FIFO queues eliminate duplicates through content-based deduplication or explicit deduplication IDs, reducing consumer complexity.
How FIFO Queues Work Internally
A FIFO queue is identified by its name ending with .fifo (e.g., my-queue.fifo). This suffix is mandatory and enforced by AWS. Internally, the queue maintains multiple ordered streams called message groups. Messages within the same message group are delivered one at a time, in order. Messages in different groups can be processed concurrently by different consumers.
#### Message Ordering Mechanism
Message Group ID: Every message sent to a FIFO queue must have a message group ID. This ID determines which group the message belongs to. All messages within the same group are delivered in the exact order they were sent. Messages in different groups can be processed in parallel.
Deduplication ID: To achieve exactly-once processing, each message can have a deduplication ID. If content-based deduplication is enabled, SQS uses the SHA-256 hash of the message body as the deduplication ID. The deduplication interval is 5 minutes. Within this 5-minute window, any message with the same deduplication ID is not accepted into the queue; instead, the original message is returned to the sender (if using the SendMessage API) or the duplicate is silently dropped.
Receive Message: When a consumer calls ReceiveMessage, it receives one or more messages from the queue. However, within a message group, only one message is visible at a time. The consumer must process and delete the current message before the next message in that group becomes visible. This is enforced by the visibility timeout.
Visibility Timeout: When a message is received, it becomes hidden from other consumers for the duration of the visibility timeout (default 30 seconds, min 0, max 12 hours). If the consumer does not delete the message within that time, it becomes visible again and can be re-received. In FIFO queues, the visibility timeout also blocks the next message in the same group from being delivered until the current message is deleted or the timeout expires.
Key Components, Values, Defaults, and Timers
Queue Name Suffix: Must end with .fifo. Example: orders.fifo.
Message Group ID: Required. It is a string of up to 128 characters. Messages with the same group ID are delivered in order.
Deduplication ID: Optional if content-based deduplication is enabled. Maximum 128 characters. If not provided and content-based deduplication is enabled, SQS uses the message body hash.
Content-Based Deduplication: An attribute you can enable when creating the queue. When enabled, SQS automatically uses the SHA-256 hash of the message body as the deduplication ID. If disabled, you must provide a deduplication ID explicitly.
Deduplication Interval: 5 minutes. This is the time window during which duplicates are rejected.
Visibility Timeout: Default 30 seconds. Range: 0 seconds to 12 hours. Must be set to a value that allows your consumer to process and delete the message.
Receive Message Wait Time: Used for long polling. Default 0 seconds. Maximum 20 seconds.
Maximum Message Size: 256 KB (including base64 encoding of binary data).
Message Retention Period: Default 4 days. Range: 1 minute to 14 days.
Delivery Delay: Default 0 seconds. Maximum 15 minutes. Not commonly used with FIFO.
Throughput Limits: FIFO queues support up to 300 transactions per second (TPS) per API action. For SendMessage, ReceiveMessage, and DeleteMessage, the combined throughput is 300 TPS per queue. With batching (up to 10 messages per batch), you can achieve up to 3,000 messages per second. However, these limits apply per queue, not per message group. If you need higher throughput, you can use multiple message groups or request a limit increase.
Configuration and Verification Commands
You can create a FIFO queue using the AWS Management Console, AWS CLI, or SDK. Below are examples using the AWS CLI.
Create a FIFO Queue:
aws sqs create-queue --queue-name my-queue.fifo --attributes \
"FifoQueue=true,ContentBasedDeduplication=true"Send a Message:
aws sqs send-message --queue-url https://sqs.us-east-1.amazonaws.com/123456789012/my-queue.fifo \
--message-body "Order #123" \
--message-group-id "group1" \
--message-deduplication-id "dedup123"Receive Messages:
aws sqs receive-message --queue-url https://sqs.us-east-1.amazonaws.com/123456789012/my-queue.fifo \
--max-number-of-messages 10 \
--wait-time-seconds 20Delete a Message:
aws sqs delete-message --queue-url https://sqs.us-east-1.amazonaws.com/123456789012/my-queue.fifo \
--receipt-handle "AQEB..."How FIFO Interacts with Related Technologies
AWS Lambda: Lambda can poll SQS FIFO queues as an event source. When using FIFO queues with Lambda, the concurrency is limited to 1 per message group. This means if you have multiple message groups, Lambda can process them in parallel, but within a group, messages are processed one at a time. The batch size for FIFO queues is limited to 10 (the same as standard).
Amazon SNS: SNS can send messages to SQS FIFO queues as a subscriber. However, the topic must be a FIFO topic (ending with .fifo). This enables ordered, deduplicated fan-out patterns.
AWS Step Functions: Step Functions can integrate with SQS FIFO queues for task states, allowing ordered execution of steps.
Amazon Kinesis: Kinesis data streams provide ordering within a shard, but they are designed for real-time streaming at high throughput. FIFO queues are better suited for decoupling microservices with strict ordering requirements at lower throughput.
Common Pitfalls and Exam Traps
Mixing Standard and FIFO: You cannot convert a standard queue to FIFO or vice versa. You must delete and recreate.
Missing .fifo suffix: The queue name must end with .fifo. The API will reject creation otherwise.
Forgetting Message Group ID: All messages sent to a FIFO queue must include a message group ID. The exam may test that you cannot send a message without it.
Deduplication ID Reuse: If you reuse a deduplication ID after 5 minutes, the message will be accepted as a new message. The deduplication window is strictly 5 minutes.
Throughput Limits: The 300 TPS limit is per API action. Batching allows you to send up to 10 messages per API call, achieving 3,000 messages per second. However, the 300 TPS limit still applies to the number of API calls.
Visibility Timeout and Ordering: If a consumer fails to delete a message within the visibility timeout, the message becomes visible again, and the next message in the group will not be delivered until the first one is processed. This can cause head-of-line blocking.
Step-by-Step Walkthrough of a FIFO Message Lifecycle
Producer Sends a Message: The producer calls SendMessage with the message body, message group ID, and optionally a deduplication ID. SQS checks the deduplication ID against recently received messages (within the last 5 minutes). If a duplicate is found, the request succeeds but the message is not enqueued; the original message's ID is returned. Otherwise, the message is stored in the queue with a sequence number.
Message is Stored: SQS assigns a sequence number to the message, ensuring ordering within the group. The message is persisted across multiple Availability Zones for durability.
Consumer Receives the Message: The consumer calls ReceiveMessage. SQS returns the first available message from each message group that is not currently in-flight (i.e., not hidden by a visibility timeout). The message becomes hidden for the duration of the visibility timeout.
Consumer Processes the Message: The consumer processes the message (e.g., updates a database). The processing must be idempotent, although FIFO reduces the chance of duplicates.
Consumer Deletes the Message: The consumer calls DeleteMessage with the receipt handle. This permanently removes the message from the queue. Once deleted, the next message in the same group becomes visible.
Visibility Timeout Expiry: If the consumer fails to delete the message within the visibility timeout, the message becomes visible again. It can be re-received by any consumer. This may cause the same message to be processed again, but FIFO ensures that the next message in the group is not delivered until the current one is deleted.
Deduplication Window Expiry: After 5 minutes, the deduplication ID is cleared from memory. A new message with the same ID will be accepted.
Producer Sends Message with Group ID
The producer calls the SQS SendMessage API with the queue URL, message body, message group ID, and optionally a deduplication ID. SQS first validates that the queue is FIFO and that the message group ID is present. It then checks the deduplication ID against a cache of IDs received in the last 5 minutes. If a match is found, the API returns success with the original message's ID, but the new message is not enqueued. If no match, the message is assigned a sequence number and stored in the queue's ordered log for that group. The message is replicated across multiple Availability Zones.
Message Stored and Ordered Within Group
SQS internally maintains a separate ordered list for each message group ID. The sequence number is a monotonically increasing integer that determines the order. Messages are not visible to consumers until they are fully persisted. The queue's metadata records the next sequence number to deliver for each group. The deduplication ID is stored with a timestamp for the 5-minute window. The maximum message size is 256 KB. If the message exceeds this, the API returns an error.
Consumer Receives First Available Message
The consumer calls ReceiveMessage with the queue URL and optional parameters like MaxNumberOfMessages (up to 10) and WaitTimeSeconds (for long polling). SQS scans each message group for the next available message that is not in-flight (visibility timeout not active). For each group, only one message can be in-flight at a time. The message is returned with a receipt handle, a unique identifier for the receive operation. The message becomes invisible for the duration of the visibility timeout (default 30 seconds). The consumer must use the receipt handle to delete or change the visibility of the message.
Consumer Processes and Deletes Message
The consumer processes the message (e.g., updates a database, calls an external API). It must then call DeleteMessage with the receipt handle to remove the message from the queue. This signals SQS that the message has been successfully processed. Only after deletion does the next message in the same group become visible. If the consumer fails to delete, the message becomes visible again after the visibility timeout expires. The consumer should handle idempotency in case of re-delivery.
Next Message Becomes Visible
After the current message is deleted, SQS marks the next message in the same group as available for delivery. If there are multiple consumers, only one will receive it. The visibility timeout for the new message starts when it is received. This step-by-step delivery ensures strict ordering within the group. If the visibility timeout expires without deletion, the current message is re-queued, and the next message remains blocked until the current one is successfully processed and deleted.
Enterprise Scenario 1: Financial Transaction Processing
A large bank uses SQS FIFO to process debit and credit transactions for customer accounts. Each customer account is assigned a unique message group ID (e.g., account number). This ensures that all transactions for a single account are processed in order. A producer (e.g., a web application) sends a message with the transaction details, the account number as the group ID, and a deduplication ID based on the transaction ID. A fleet of EC2 instances polls the queue and processes transactions. The system is configured with a visibility timeout of 60 seconds to allow for database updates. If a consumer crashes, the message becomes visible again, and another consumer picks it up. The deduplication ID prevents double processing if the producer retries. The queue handles about 200 TPS, well within the 300 TPS limit. The bank ensures that the queue name ends with .fifo and content-based deduplication is disabled to allow explicit deduplication IDs.
Enterprise Scenario 2: E-commerce Order Management
An e-commerce platform uses FIFO queues to manage order fulfillment. Each order is assigned a unique order ID, and the message group ID is set to the customer ID. This ensures that all orders from the same customer are processed in the sequence they were placed. The system uses a Lambda function as the consumer, with a batch size of 1 to maintain order per group. The Lambda processes the order (e.g., updates inventory, sends confirmation email). The visibility timeout is set to 30 seconds. The queue is configured with content-based deduplication enabled, so if the same order message is sent twice within 5 minutes, the duplicate is rejected. The platform experiences peak traffic of 1,000 orders per second, so they use multiple message groups (different customer IDs) to achieve higher throughput. They also use batching to send up to 10 messages per API call, reducing the number of API calls.
Common Misconfiguration and Issues
Head-of-Line Blocking: If a consumer takes too long to process a message, the visibility timeout expires, and the message is re-queued. This blocks subsequent messages in the same group. To avoid this, set the visibility timeout appropriately and implement dead-letter queues for messages that repeatedly fail.
Incorrect Deduplication ID: If the producer generates a unique deduplication ID for each send attempt (e.g., including a timestamp), the deduplication mechanism will not prevent duplicates. The deduplication ID should be based on the business-level unique identifier (e.g., transaction ID).
Throughput Bottlenecks: If all messages use the same group ID, throughput is limited to one message at a time. Use multiple group IDs to allow parallel processing.
Queue Name Without .fifo: Creating a queue without the .fifo suffix results in a standard queue, losing ordering and deduplication guarantees.
DVA-C02 Exam Focus on SQS FIFO Queues
The exam tests your understanding of FIFO queue mechanics, especially the distinctions from standard queues. Key objective codes: Domain 1 (Development with AWS Services), Objective 1.5 (Decouple application components). Expect scenario-based questions where you must choose the right queue type.
Common Wrong Answers and Why Candidates Choose Them
Choosing standard queues when ordering is required: Candidates often see "high throughput" and pick standard queues, ignoring the ordering requirement. Remember: if the question mentions "order must be preserved" or "messages must be processed in sequence," FIFO is mandatory.
Assuming FIFO queues support at-least-once delivery: FIFO provides exactly-once processing, but only within the deduplication window. If a message is not deleted, it can be re-delivered. The exam may trick you by saying "FIFO guarantees exactly-once delivery" — it does, but only if the consumer deletes the message. Re-delivery can happen.
Forgetting the message group ID requirement: A question might describe sending a message to a FIFO queue without a group ID. The API will reject it. Candidates may think it's optional because it's optional for standard queues.
Confusing deduplication ID with message group ID: The deduplication ID prevents duplicates; the group ID maintains ordering. The exam may ask which ID is used for ordering (group ID) vs. deduplication.
Specific Numbers and Terms to Memorize
300 TPS per API action (SendMessage, ReceiveMessage, DeleteMessage) per queue.
Up to 3,000 messages per second with batching (10 messages per batch).
5-minute deduplication interval.
Maximum message size: 256 KB.
Default visibility timeout: 30 seconds.
Queue name must end with `.fifo`.
Message group ID: Required, up to 128 characters.
Deduplication ID: Optional if content-based deduplication enabled, up to 128 characters.
Maximum batch size: 10 messages.
Long polling wait time: Up to 20 seconds.
Edge Cases and Exceptions
Exactly-once is not absolute: If a consumer receives a message and then crashes before deleting, the message becomes visible again after the visibility timeout. The message will be re-delivered, breaking exactly-once. The exam may test that you need idempotent consumers even with FIFO.
FIFO queues cannot be used with Lambda event source mapping if you need high concurrency: Lambda concurrency per group is 1. If you need parallel processing, use multiple groups.
SNS FIFO topics: To send messages from SNS to SQS FIFO, the SNS topic must also be FIFO (name ending with .fifo).
No support for message timers or delay queues with FIFO?: Actually, FIFO queues do support delivery delay (0 seconds to 15 minutes), but it's rarely tested.
How to Eliminate Wrong Answers
If the question mentions "order" or "sequence," eliminate standard queue options.
If it mentions "duplicate messages" or "exactly-once," FIFO is likely the answer.
If the queue name in the question does not end with .fifo, it's a standard queue.
If the question says "high throughput" without ordering, standard is better.
For throughput, remember that FIFO has lower throughput (300 TPS) compared to standard (nearly unlimited).
FIFO queues guarantee strict ordering within a message group and exactly-once processing via deduplication.
Queue name must end with `.fifo`; otherwise, it is a standard queue.
Message group ID is required for every message sent to a FIFO queue.
Deduplication ID prevents duplicates within a 5-minute window; can be content-based or explicit.
Maximum throughput is 300 TPS per API action (3,000 messages/second with batching up to 10).
Visibility timeout blocks subsequent messages in the same group until the current message is deleted or times out.
FIFO queues cannot be converted from standard queues; you must create a new queue.
Lambda concurrency with FIFO is 1 per message group; use multiple groups for parallel processing.
SNS topics must be FIFO to send messages to SQS FIFO queues.
Dead-letter queues can be used for messages that fail processing repeatedly.
These come up on the exam all the time. Here's how to tell them apart.
SQS Standard Queue
Best-effort ordering (messages may arrive out of order).
At-least-once delivery (duplicates possible).
Nearly unlimited throughput (thousands of TPS).
No message group ID required.
Queue name can be any valid name (no suffix requirement).
SQS FIFO Queue
Strict first-in-first-out ordering within a message group.
Exactly-once processing (within deduplication window).
Limited to 300 TPS per API action (3,000 with batching).
Message group ID required for every message.
Queue name must end with `.fifo`.
Mistake
FIFO queues guarantee exactly-once delivery in all cases.
Correct
FIFO queues provide exactly-once processing only if the consumer deletes the message before the visibility timeout expires. If the consumer crashes or the timeout expires, the message can be re-delivered. Exactly-once is achieved through deduplication on send, not on receive. Consumers should still be idempotent.
Mistake
All messages in a FIFO queue are delivered in strict order globally.
Correct
Ordering is guaranteed only within the same message group. Messages in different groups can be delivered out of order relative to each other. The exam may test that you need to assign the same group ID to maintain order across all messages.
Mistake
FIFO queues support the same throughput as standard queues.
Correct
FIFO queues have a maximum throughput of 300 TPS per API action (3,000 messages per second with batching), whereas standard queues can handle nearly unlimited throughput. This is a key trade-off.
Mistake
You can convert a standard queue to a FIFO queue by changing attributes.
Correct
You cannot convert a queue type. You must delete the standard queue and create a new FIFO queue with the `.fifo` suffix. The exam may test that the queue name must end with `.fifo` at creation time.
Mistake
Deduplication ID is required for every message in a FIFO queue.
Correct
If content-based deduplication is enabled, the deduplication ID is optional; SQS uses the SHA-256 hash of the message body. If disabled, you must provide a deduplication ID. The exam may ask which scenario requires an explicit deduplication ID.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
The send operation will fail with an error. The message group ID is required for all messages sent to a FIFO queue. The API will return a `MissingParameter` error. Always include a message group ID, even if you only have one group.
No. You cannot change the queue type after creation. You must delete the standard queue and create a new FIFO queue with a name ending in `.fifo`. This is a common exam trap: candidates might think they can update the `FifoQueue` attribute.
When content-based deduplication is enabled, SQS computes the SHA-256 hash of the message body and uses it as the deduplication ID. If you send two messages with identical bodies within 5 minutes, the second send will succeed but the message will not be enqueued; the original message's ID is returned. This is useful for retries without explicit deduplication IDs.
The maximum throughput is 300 transactions per second (TPS) per API action (SendMessage, ReceiveMessage, DeleteMessage). With batching (up to 10 messages per batch), you can achieve up to 3,000 messages per second. However, the 300 TPS limit applies to the number of API calls, not messages. If you need higher throughput, you can request a limit increase from AWS.
Yes, Lambda can poll SQS FIFO queues as an event source. However, the concurrency is limited to 1 per message group. This means that if you have multiple message groups, Lambda can process them in parallel, but within a group, messages are processed one at a time. The batch size is limited to 10 (the same as standard queues).
The deduplication window is 5 minutes. Within this 5-minute period, any message with the same deduplication ID (or same body if content-based deduplication is enabled) will be rejected as a duplicate. After 5 minutes, the deduplication ID is cleared, and a new message with the same ID will be accepted.
In FIFO queues, only one message per message group can be in-flight (i.e., received but not yet deleted) at a time. The visibility timeout determines how long a message is hidden after being received. If the consumer does not delete the message within the visibility timeout, it becomes visible again, and the next message in the group remains blocked until the current one is deleted. This can cause head-of-line blocking if the consumer takes too long.
You've just covered SQS FIFO Queues: Ordering and Deduplication — now see how well it sticks with free DVA-C02 practice questions. Full explanations included, no account needed.
Done with this chapter?