DEA-C01 Data Operations and Support Practice Question
A company uses Amazon EMR to run Spark jobs on data stored in S3. After upgrading the EMR cluster to a new release, one of the Spark jobs fails with 'OutOfMemoryError' in the executor. Which configuration change is MOST likely to resolve this issue?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Increase spark.executor.memory to allocate more memory per executor.
Increasing spark.executor.memory directly allocates more memory per executor, addressing the OutOfMemoryError in the executor. Option A (increasing the number of core nodes) adds more cluster capacity but does not increase the memory available to individual executors, so it may not resolve the OOM if the existing executors are already memory-constrained. Option B (decreasing spark.sql.shuffle.partitions) reduces the number of shuffle partitions, which can increase the size of each partition and potentially cause more memory pressure, not less. Option C (increasing spark.driver.memory) only helps the driver process, not the executor, so it does not fix executor OOM errors.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Increase the number of core nodes in the EMR cluster.
Why it's wrong here
Adding core nodes increases cluster capacity and parallelism but does not enlarge each executor's heap, so the same per-executor OutOfMemoryError recurs. Scaling out is correct when the cluster is CPU-bound or under-provisioned, not when individual executors exhaust memory.
- ✗
Decrease spark.sql.shuffle.partitions to reduce overhead.
Why it's wrong here
Fewer shuffle partitions means each task processes a larger slice of data, increasing per-task memory pressure and worsening the executor OutOfMemoryError. Reducing partitions is correct when many tiny tasks cause scheduling overhead, not when executors exhaust heap during shuffles.
- ✗
Increase spark.driver.memory in the Spark configuration.
Why it's wrong here
The error occurs in the executor, so raising driver memory leaves the failing JVM unchanged; executor heap is set by spark.executor.memory. Driver memory is the right lever when the driver itself throws OutOfMemoryError, for example during large collect or broadcast operations.
- ✓
Increase spark.executor.memory to allocate more memory per executor.
Why this is correct
Spark executor memory is set by spark.executor.memory; the OutOfMemoryError arises because the new EMR release changed default executor sizing, leaving too little heap. Raising this value directly expands executor heap, satisfying the job's memory constraint without altering cluster hardware.
Visual reference
Quick reference
AWS S3 Storage Class Comparison
| Storage Class | Min Duration | Retrieval | Use Case |
|---|---|---|---|
| S3 Standard | None | Immediate | Frequently accessed data |
| S3 Standard-IA | 30 days | Immediate | Infrequent access, rapid retrieval |
| S3 One Zone-IA | 30 days | Immediate | Non-critical infrequent data |
| S3 Intelligent-Tiering | None | Immediate–hours | Unknown or changing access patterns |
| S3 Glacier Instant | 90 days | Milliseconds | Archive with instant retrieval |
| S3 Glacier Flexible | 90 days | Minutes–hours | Archive, flexible retrieval |
| S3 Glacier Deep Archive | 180 days | Hours | Long-term compliance archive |
Go deeper
Related to this question
About these practice questions
One of 1,321 original DEA-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This DEA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DEA-C01 exam.