DEA-C01 Data Ingestion and Transformation Practice Question
A financial services company processes real-time stock trade data. They use Amazon Kinesis Data Streams with a shard count of 5, each shard receiving about 500 records per second. The consumer application uses the Kinesis Client Library (KCL) with DynamoDB for checkpointing. Lately, some records are being processed multiple times. What is the most likely cause?
⚠ Common exam trap
Candidates often confuse checkpoint throttling (Option C) with duplicate processing, but throttling would cause checkpoint failures and potential re-processing only if the application cannot recover, whereas the direct cause of duplicates is the gap between processing and checkpointing after a crash.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
The consumer application is crashing and restarting, causing re-processing of records.
The Kinesis Client Library (KCL) uses DynamoDB to track checkpoint progress for each shard. If the consumer application crashes and restarts, the KCL will resume processing from the last committed checkpoint, which may be behind the actual processing point. This causes records that were already processed (but not yet checkpointed) to be re-processed, leading to duplicate processing.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
The consumer application is crashing and restarting, causing re-processing of records.
Why this is correct
Frequent crashes force the KCL to resume from the last DynamoDB checkpoint, replaying every record consumed after it — at-least-once delivery guarantees duplicates on restart. With five shards at 500 records per second, each restart re-processes a substantial backlog, matching the observed duplicate processing.
- ✗
The Kinesis stream's iterator age is exceeding the retention period.
Why it's wrong here
Exceeding retention causes data loss through expired iterators, not duplicate processing; KCL resumes from the last DynamoDB checkpoint. Iterator age is the metric to watch for consumer lag against the 24-hour default retention, a different failure mode entirely.
- ✗
The DynamoDB table used for checkpointing is throttling write requests.
Why it's wrong here
Checkpoint throttling delays writes to the Kinesis Client Library's lease and checkpoint table, so the consumer re-reads records from an earlier sequence number, causing duplicates. It is tempting because DynamoDB throttling is a common operational fault, and it would be the correct cause if the application were failing to persist checkpoints at all rather than processing them twice.
- ✗
The record size exceeds the 1 MB API limit, causing retries.
Why it's wrong here
Records exceeding 1 MB are rejected by PutRecords with a size error, not silently reprocessed; KCL duplicate delivery stems from checkpointing behaviour. The 1 MB limit matters when designing producer batching, not when diagnosing at-least-once reprocessing.
Go deeper
Related to this question
About these practice questions
This DEA-C01 question is part of Courseiva's 1,321-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This DEA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DEA-C01 exam.