MLS-C01 Data Engineering Practice Question

Correct answer & explanation

Using DynamoDB conditional writes with the Kinesis sequence number as a unique attribute ensures idempotency. When processing a record, the Lambda function can attempt a conditional write that only succeeds if an item with that sequence number does not already exist. If the write fails due to a condition check, it means the record was already processed, so the function can skip it. This achieves exactly-once semantics even if the same record is delivered multiple times due to retries. Option A is incorrect because an SQS FIFO queue between Kinesis and Lambda would add latency and complexity; Kinesis already provides ordering and at-least-once delivery, and using a FIFO queue does not guarantee that the Lambda function will not process the same record multiple times within its retry logic. Option B is incorrect because setting maximum retry count to 0 with a DLQ simply discards failed records after the first failure, which does not provide exactly-once processing; it results in at-most-once semantics and potential data loss. Option C is incorrect because increasing DynamoDB write capacity may reduce throttling but does not eliminate duplicate processing; the same record can still be retried and written multiple times if the function retries, leading to duplicate writes.