DEA-C01 Data Operations and Support Practice Question

A company runs an Amazon EMR cluster with Spark jobs that process data from Amazon S3. The data engineer receives an alert that one of the Spark jobs failed with an OutOfMemoryError. The job processes large files and uses the default Spark configurations. Which configuration change is MOST likely to resolve the issue?

⚠ Common exam trap

The trap is assuming that adding more executors (horizontal scaling) fixes a memory error, when OOM is a per-executor heap problem that requires increasing executor memory or fixing data skew.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Increase the spark.executor.memory configuration.

An OutOfMemoryError in Spark typically occurs when an executor's JVM heap is exhausted during processing of large partitions or files. Increasing spark.executor.memory directly expands the heap available to each executor, allowing it to hold more data in memory and resolve the OOM condition.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✓
Increase the spark.executor.memory configuration.
Why this is correct
Spark executors hold partitions in heap, so an OutOfMemoryError during processing of large files signals insufficient executor heap. Raising spark.executor.memory directly expands that heap, letting each executor hold its partition data without spilling or failing, which addresses the default-configuration constraint in the stem.
✗
Increase the number of executors.
Why it's wrong here
Adding executors increases parallelism but each executor still holds the same default heap, so a single oversized partition or task still exceeds memory. It is tempting because more executors spread partitions across the cluster, which would be correct when the workload is bottlenecked by too few concurrent tasks rather than by per-task memory.
✗
Disable dynamic resource allocation.
Why it's wrong here
Dynamic allocation already returns idle executors and cannot cause a single executor to exhaust its heap while processing large files; disabling it removes elasticity without changing per-task memory. It is tempting because allocation churn can trigger executor loss, which would be the correct fix when executors are being pre-empted mid-shuffle on a shared cluster.
✗
Decrease the number of cores per executor.
Why it's wrong here
Fewer cores per executor reduces concurrent task slots, so each task receives a smaller share of the executor heap, worsening the OutOfMemoryError on large files. It is tempting because reducing parallelism can lower memory pressure from too many simultaneous tasks, which would be correct when executors are over-subscribed with many small partitions.

Visual reference

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

Go deeper

DEA-C01 Data Operations and Support Practice Question

Why each option matters

Option-by-option breakdown

AWS S3 Storage Class Comparison

Related to this question