DEA-C01 Data Operations and Support Practice Question

A data pipeline uses AWS Glue to process data from Amazon S3. The job fails with an 'OutOfMemoryError' during the transformation phase. Which action should the data engineer take to resolve this issue?

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Increase the number of DPUs (Data Processing Units) for the Glue job.

The OutOfMemoryError occurs because the Glue job does not have enough memory allocated. Increasing the number of DPUs (Data Processing Units) increases both memory and processing capacity, directly resolving the issue. Option A (S3 server-side encryption) affects data security, not memory. Option B (increasing data partitions) may help parallelism but does not directly increase memory per executor. Option C (changing to Parquet) can reduce data volume but does not guarantee sufficient memory for transformation.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Enable S3 server-side encryption.
Why it's wrong here
Server-side encryption protects data at rest and has no bearing on executor heap consumption, so it cannot prevent an OutOfMemoryError during transformation. It tempts because encryption is a common S3 security control, yet it would be the correct answer only if the requirement were compliance or data-protection, not memory exhaustion.
✗
Increase the number of partitions in the input data.
Why it's wrong here
Adding input partitions increases the number of small files and tasks, raising per-task overhead and shuffle pressure rather than freeing executor memory. It tempts because partitioning is a genuine Glue tuning lever, yet it would be correct when the problem is skewed, oversized partitions causing a single task to spill, not overall heap exhaustion.
✗
Change the data format from CSV to Parquet.
Why it's wrong here
Converting CSV to Parquet reduces storage and scan volume, but the OutOfMemoryError arises during transformation, where the executor heap is exhausted; columnar layout does not shrink in-memory shuffle or aggregation state. It tempts because Parquet is the standard Glue performance recommendation, yet it would be right when the bottleneck is S3 read throughput or crawler cost.
✓
Increase the number of DPUs (Data Processing Units) for the Glue job.
Why this is correct
Adding DPUs increases the total executor memory and parallelism available to the Glue job, letting Spark distribute transformation partitions across more workers. This directly addresses the OutOfMemoryError caused by insufficient memory per executor during the transformation phase.

Visual reference

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

Go deeper

DEA-C01 Data Operations and Support Practice Question

Why each option matters

Option-by-option breakdown

AWS S3 Storage Class Comparison

Related to this question