MLS-C01 Data Engineering Practice Question
A company uses Amazon EMR to run Spark jobs on a large dataset stored in Amazon S3. The jobs are failing with 'OutOfMemoryError' in the executors. The data is not skewed. Which configuration change will most likely resolve the issue?
⚠ Common exam trap
Test-takers frequently confuse executor memory (heap) with memoryOverhead (off-heap), assuming that increasing heap or reducing partitions will fix all OutOfMemoryErrors, when in fact shuffle-heavy workloads require explicit off-heap tuning.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Increase the spark.executor.memoryOverhead setting
When Spark executors run out of memory during shuffle operations, the `spark.executor.memoryOverhead` setting is often the culprit. This parameter allocates off-heap memory for JVM overhead, internal metadata, and shuffle buffers. Increasing it provides more room for these operations without reducing the executor heap, directly addressing OutOfMemoryError in non-skewed data scenarios.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Enable Kryo serialization
Why it's wrong here
Kryo reduces memory usage but may not be sufficient to resolve OOM if memory overhead is too low.
- ✗
Decrease the number of shuffle partitions
Why it's wrong here
Fewer partitions reduce memory usage but may cause data skew and does not directly address OOM.
- ✓
Increase the spark.executor.memoryOverhead setting
Why this is correct
Memory overhead handles JVM overhead and off-heap memory, preventing OOM errors.
- ✗
Increase the number of executor cores
Why it's wrong here
More cores can increase parallelism but not memory per executor.
Visual reference
Quick reference
AWS S3 Storage Class Comparison
| Storage Class | Min Duration | Retrieval | Use Case |
|---|---|---|---|
| S3 Standard | None | Immediate | Frequently accessed data |
| S3 Standard-IA | 30 days | Immediate | Infrequent access, rapid retrieval |
| S3 One Zone-IA | 30 days | Immediate | Non-critical infrequent data |
| S3 Intelligent-Tiering | None | Immediate–hours | Unknown or changing access patterns |
| S3 Glacier Instant | 90 days | Milliseconds | Archive with instant retrieval |
| S3 Glacier Flexible | 90 days | Minutes–hours | Archive, flexible retrieval |
| S3 Glacier Deep Archive | 180 days | Hours | Long-term compliance archive |
Go deeper
Related to this question
About these practice questions
This MLS-C01 question is part of Courseiva's 1,672-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.