A startup is designing a cloud-native application that processes IoT sensor data. The data arrives in bursts, and processing must be fault-tolerant with exactly-once semantics. The team considers Apache Kafka, RabbitMQ, and Amazon SQS. Which choice best meets the requirements of a cloud-native architecture?
Trap 1: Use Amazon SQS with FIFO queues for ordering and deduplication.
SQS FIFO provides exactly-once but is a proprietary service, deviating from cloud-native portability principles.
Trap 2: Use RabbitMQ with publisher confirms and consumer acknowledgements.
RabbitMQ's at-least-once semantics don't guarantee exactly-once processing.
Trap 3: Implement an HTTP endpoint that the IoT devices call directly.
HTTP callbacks are unreliable and don't guarantee delivery or exactly-once processing.
- A
Use Apache Kafka with idempotent producers and transactional APIs.
Kafka's transactional support ensures exactly-once semantics, and its log-based architecture handles bursty data well.
- B
Use Amazon SQS with FIFO queues for ordering and deduplication.
Why wrong: SQS FIFO provides exactly-once but is a proprietary service, deviating from cloud-native portability principles.
- C
Use RabbitMQ with publisher confirms and consumer acknowledgements.
Why wrong: RabbitMQ's at-least-once semantics don't guarantee exactly-once processing.
- D
Implement an HTTP endpoint that the IoT devices call directly.
Why wrong: HTTP callbacks are unreliable and don't guarantee delivery or exactly-once processing.