MLS-C01 Data Engineering Practice Question
A company runs an e-commerce platform that generates clickstream data in real-time. The data is ingested into Amazon Kinesis Data Streams (100 shards) and processed by AWS Lambda functions, which aggregate data in 1-minute windows and write the results to Amazon S3. The Lambda functions are triggered by the Kinesis stream using the event source mapping. Recently, the company noticed that some records are being processed multiple times, leading to duplicate data in S3. The Lambda function is idempotent, but the duplicates are causing downstream issues. The Lambda function's concurrency limit is 1000, and the batch size is 100. The average processing time per record is 200 ms. What is the most likely cause of the duplicates, and how should it be fixed?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Ensure the Lambda function is idempotent and uses the sequence number to deduplicate records.
Lambda functions process records from Kinesis in batches. If the function fails (e.g., due to timeout or error), the entire batch is retried, causing duplicates if some records were already partially processed. To avoid duplicates, the function should be idempotent and should not commit partial results. Option A is wrong because the concurrency is sufficient. Option C is wrong because increasing batch size increases the risk of partial failure. Option D is wrong because a FIFO queue does not integrate with Kinesis.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Increase the Lambda concurrency limit to 2000 to handle the load.
Why it's wrong here
Concurrency is not the issue; duplicates are caused by retries on failure.
- ✓
Ensure the Lambda function is idempotent and uses the sequence number to deduplicate records.
Why this is correct
If the function fails and retries, using sequence numbers allows it to skip already processed records, preventing duplicates.
- ✗
Decrease the batch size to 10 to reduce the impact of failures.
Why it's wrong here
Smaller batch size reduces the number of records affected per failure but does not eliminate duplicates if the function is not idempotent.
- ✗
Use Amazon SQS FIFO queue as a buffer between Kinesis and Lambda to guarantee exactly-once processing.
Why it's wrong here
Kinesis does not directly integrate with SQS; this adds complexity and latency.
Quick reference
AWS S3 Storage Class Comparison
| Storage Class | Min Duration | Retrieval | Use Case |
|---|---|---|---|
| S3 Standard | None | Immediate | Frequently accessed data |
| S3 Standard-IA | 30 days | Immediate | Infrequent access, rapid retrieval |
| S3 One Zone-IA | 30 days | Immediate | Non-critical infrequent data |
| S3 Intelligent-Tiering | None | Immediate–hours | Unknown or changing access patterns |
| S3 Glacier Instant | 90 days | Milliseconds | Archive with instant retrieval |
| S3 Glacier Flexible | 90 days | Minutes–hours | Archive, flexible retrieval |
| S3 Glacier Deep Archive | 180 days | Hours | Long-term compliance archive |
Go deeper
Related to this question
About these practice questions
This MLS-C01 question is part of Courseiva's 1,672-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.