An order-processing service consumes messages from an Amazon SQS Standard queue using a custom worker. During traffic spikes, the worker occasionally times out after performing some work but before acknowledging the message, so SQS redelivers it and it may be processed again.
You also observe that a small set of “poison” messages always fail validation.
What change most directly improves resilience by (1) preventing poison messages from retrying indefinitely and (2) avoiding duplicate side effects caused by legitimate retries?
Trap 1: Increase the SQS visibility timeout and, when validation fails,…
Increasing the visibility timeout only gives the consumer more time to process, temporarily reducing the chance of redelivery while a message is in flight, but it does not prevent the retry loop for a message that consistently fails to validate or process. If the consumer calls DeleteMessage after validation failure, you are permanently discarding the poison message without any opportunity to inspect, log, or route it for remediation, which undermines operational debugging and can silently drop legitimate business events. SQS's actual poison-message solution is a dead-letter queue with a redrive policy that moves a message to a DLQ after the configured maxReceiveCount, allowing for isolation and recovery. Since this option lacks that DLQ-based quarantine and also does not implement idempotency to handle at-least-once redelivery of valid messages, it is insufficient.
Trap 2: Move to SNS topics with subscriptions and rely on SNS to provide…
SNS is a pub/sub messaging service that does not offer exactly-once delivery semantics; its standard topic type delivers messages at least once and can produce duplicates during retries or due to network issues. Subscribers (e.g., SQS queues) still need their own idempotency handling because SNS does not deduplicate messages for you. Moreover, switching to SNS does not address the core problem of poison messages: SNS lacks a built-in dead-letter queue with redrive policies like SQS, so you'd still need to design a quarantine mechanism separately. Therefore, this option fails to provide the required duplicate protection and poison-message handling.
Trap 3: Change the queue to FIFO and enable content-based deduplication,…
FIFO with content-based deduplication may reduce some duplicates, but it does not guarantee protection against duplicate side effects when the consumer times out or fails after partially processing. Poison-message retry loops still need a DLQ/redrive approach, and idempotency is still required to make processing safe under retries.
- A
Increase the SQS visibility timeout and, when validation fails, call DeleteMessage in the consumer to remove the message immediately.
Why wrong: Increasing the visibility timeout only gives the consumer more time to process, temporarily reducing the chance of redelivery while a message is in flight, but it does not prevent the retry loop for a message that consistently fails to validate or process. If the consumer calls DeleteMessage after validation failure, you are permanently discarding the poison message without any opportunity to inspect, log, or route it for remediation, which undermines operational debugging and can silently drop legitimate business events. SQS's actual poison-message solution is a dead-letter queue with a redrive policy that moves a message to a DLQ after the configured maxReceiveCount, allowing for isolation and recovery. Since this option lacks that DLQ-based quarantine and also does not implement idempotency to handle at-least-once redelivery of valid messages, it is insufficient.
- B
Move to SNS topics with subscriptions and rely on SNS to provide exactly-once delivery to eliminate duplicates automatically.
Why wrong: SNS is a pub/sub messaging service that does not offer exactly-once delivery semantics; its standard topic type delivers messages at least once and can produce duplicates during retries or due to network issues. Subscribers (e.g., SQS queues) still need their own idempotency handling because SNS does not deduplicate messages for you. Moreover, switching to SNS does not address the core problem of poison messages: SNS lacks a built-in dead-letter queue with redrive policies like SQS, so you'd still need to design a quarantine mechanism separately. Therefore, this option fails to provide the required duplicate protection and poison-message handling.
- C
Configure a dead-letter queue (DLQ) with a redrive policy that moves messages after maxReceiveCount, and implement idempotent processing in the consumer using an idempotency key.
SQS Standard is at-least-once delivery, so timeouts can cause redelivery and duplicates. A DLQ with a redrive policy prevents poison messages from retrying forever by moving them after repeated failures. Idempotent processing (for example, storing a processed marker in a database with conditional logic keyed by an idempotency key) prevents duplicate side effects when retries occur for valid messages.
- D
Change the queue to FIFO and enable content-based deduplication, leaving the consumer logic unchanged.
Why wrong: FIFO with content-based deduplication may reduce some duplicates, but it does not guarantee protection against duplicate side effects when the consumer times out or fails after partially processing. Poison-message retry loops still need a DLQ/redrive approach, and idempotency is still required to make processing safe under retries.