DEA-C01 Data Ingestion and Transformation Practice Question

A company uses Amazon EMR to process large datasets stored in Amazon S3. The data is in Parquet format and partitioned by date. The EMR cluster uses Spark SQL for transformations. Recently, the job has been slow and some tasks are failing due to 'java.lang.OutOfMemoryError'. The cluster has 10 core nodes of type m5.xlarge. Which configuration change would MOST improve performance and stability?

⚠ Common exam trap

The trap here is that candidates often focus on tuning Spark configurations (partitions, cores, serialization) to fix OutOfMemoryErrors, but the real issue is insufficient physical memory per node, which requires a change in instance family rather than software settings.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Change the core node instance type to r5.xlarge (memory-optimized).

The error 'java.lang.OutOfMemoryError' indicates that the Spark executors are running out of memory during processing. The m5.xlarge instance type provides 16 GiB of memory, but the workload likely requires more memory per task. Switching to r5.xlarge (32 GiB of memory) doubles the available memory per node, reducing memory pressure and preventing task failures, which directly improves stability and performance for memory-intensive transformations.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Increase the number of Spark partitions using repartition(), but keep the same nodes.
Why it's wrong here
More partitions shrink each task's data slice but leave executor heap unchanged, so skewed partitions still exceed memory and spill or fail. Repartitioning suits uneven partition sizes causing stragglers. Raising executor memory or switching to a larger node type addresses the OutOfMemoryError directly.
✓
Change the core node instance type to r5.xlarge (memory-optimized).
Why this is correct
OutOfMemoryError during Spark SQL tasks indicates executor heap exhaustion, not CPU shortage. Switching core nodes to r5.xlarge increases RAM per node, giving executors more memory for shuffles and aggregations, which addresses the memory constraint causing task failures and slowness.
✗
Increase the number of executor cores in the Spark configuration.
Why it's wrong here
More cores per executor divide the same heap among concurrent tasks, reducing memory available to each and worsening OutOfMemoryError. This tuning helps CPU-bound jobs with small per-task footprints. Increasing executor memory or core node size gives each task the heap it needs.
✗
Enable Kryo serialization in the Spark configuration.
Why it's wrong here
Kryo reduces serialisation overhead and shuffle bytes, improving speed, but does not enlarge executor heap, so tasks still exhaust memory. It suits CPU- or network-bound jobs with ample memory. Raising executor memory or node size resolves the OutOfMemoryError itself.

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

Go deeper

DEA-C01 Data Ingestion and Transformation Practice Question

Why each option matters

Option-by-option breakdown

AWS S3 Storage Class Comparison

Related to this question