DEA-C01 Data Operations and Support Practice Question
A company runs a data pipeline that uses Amazon EMR to process large datasets. The pipeline reads data from S3, processes it using Spark, and writes results back to S3. Recently, the pipeline has been failing with 'OutOfMemoryError' in the Spark executors. The EMR cluster is configured with 5 core nodes of type m5.xlarge (4 vCPU, 16 GB memory each). The Spark application uses dynamic allocation and default Spark configurations. The input data size is approximately 500 GB in Parquet format. What is the most cost-effective way to resolve the out-of-memory errors?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Change the core node instance type to r5.xlarge (32 GB memory) and keep 5 nodes.
The current cluster has limited memory per node (16 GB). By switching to memory-optimized instances like r5.xlarge (32 GB), each node has double the memory, reducing the chance of OOM. This is more cost-effective than adding more nodes because the total memory per node increases without increasing the number of instances. Option A is wrong because increasing the number of nodes adds more memory but also more cost; it might be more expensive than using fewer, larger nodes. Option C is wrong because it's generally not recommended to increase spark.executor.memory beyond the physical memory; it could cause YARN to kill containers. Option D is wrong because Parquet is already efficient; changing to a different format may not solve memory issues.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Increase the spark.executor.memory setting to 8 GB in the Spark configuration.
Why it's wrong here
The default executor memory is already around 4 GB; increasing to 8 GB may exceed container limit and cause YARN to kill executors.
- ✓
Change the core node instance type to r5.xlarge (32 GB memory) and keep 5 nodes.
Why this is correct
Memory-optimized instances provide more memory per node, reducing OOM without increasing node count.
- ✗
Increase the number of core nodes to 10 to distribute the data across more executors.
Why it's wrong here
Adding nodes increases cost and may not solve OOM if each executor still has limited memory.
- ✗
Change the input data format from Parquet to ORC to reduce memory footprint.
Why it's wrong here
Both formats are columnar and similar in memory usage; format change is unlikely to fix OOM.
Visual reference
Quick reference
AWS S3 Storage Class Comparison
| Storage Class | Min Duration | Retrieval | Use Case |
|---|---|---|---|
| S3 Standard | None | Immediate | Frequently accessed data |
| S3 Standard-IA | 30 days | Immediate | Infrequent access, rapid retrieval |
| S3 One Zone-IA | 30 days | Immediate | Non-critical infrequent data |
| S3 Intelligent-Tiering | None | Immediate–hours | Unknown or changing access patterns |
| S3 Glacier Instant | 90 days | Milliseconds | Archive with instant retrieval |
| S3 Glacier Flexible | 90 days | Minutes–hours | Archive, flexible retrieval |
| S3 Glacier Deep Archive | 180 days | Hours | Long-term compliance archive |
Go deeper
Related to this question
About these practice questions
One of 1,711 original DEA-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This DEA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DEA-C01 exam.