MLS-C01 Data Engineering Practice Question
A company processes large streams of IoT sensor data using Amazon Kinesis Data Streams with 100 shards. Each sensor reading is about 1 KB. The data is consumed by an Amazon EMR cluster running Spark Streaming jobs. The team notices that the Spark Streaming job's processing time is gradually increasing, and the stream is falling behind. They suspect the issue is due to skewed data distribution across shards. Which approach should the team take to diagnose and resolve the issue?
⚠ Common exam trap
Watch out — candidates often confuse consumer-side rebalancing (KCL or Spark repartitioning) with producer-side data distribution, and incorrectly assume that increasing shards or using Spark repartitioning can fix a hot shard caused by a poor partition key.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Modify the producer to add a random prefix to the partition key, ensuring even distribution across all shards, and monitor the stream using CloudWatch.
Adding a random prefix to the partition key ensures that sensor data is evenly distributed across all 100 shards, eliminating hot shards that cause processing delays. This directly addresses the skewed data distribution issue without requiring infrastructure changes, and the team can monitor the improvement using CloudWatch metrics like IncomingBytes and ReadProvisionedThroughputExceeded.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Increase the number of shards to 200 to provide more parallelism.
Why it's wrong here
Increasing shards does not fix the skew; if data is concentrated on a few shards, the new shards may remain underutilized.
- ✓
Modify the producer to add a random prefix to the partition key, ensuring even distribution across all shards, and monitor the stream using CloudWatch.
Why this is correct
Adding a random prefix to partition keys uniformizes distribution, eliminating hot shards; CloudWatch helps confirm the fix.
- ✗
Check Amazon CloudWatch metrics for Kinesis to identify hot shards, then manually redistribute the data by repartitioning in Spark.
Why it's wrong here
CloudWatch can identify hot shards, but manual repartitioning in Spark after ingestion does not resolve the root cause (skew in partition keys).
- ✗
Use the Kinesis Client Library (KCL) with a custom worker to rebalance the load across shards.
Why it's wrong here
KCL is not typically used with Spark Streaming; Spark has its own Kinesis connector.
Visual reference
Go deeper
Related to this question
About these practice questions
One of 1,672 original MLS-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.