How to Fix AWS Glue Out-of-Memory Errors in ETL Jobs
A company is using AWS Glue to run ETL jobs that transform data from S3 to Redshift. The jobs are failing intermittently with out-of-memory errors. Which THREE actions can help resolve this issue? (Choose THREE.)
Quick Answer
The answer is to increase DPUs, use a larger worker type like G.2X, and optimize transformation logic. These three actions directly resolve out-of-memory errors by either allocating more memory per worker or reducing the memory footprint of the job itself. Increasing the number of DPUs adds total compute and memory capacity, while switching to a larger worker type like G.2X boosts memory per executor, which is critical when processing skewed or large datasets. Optimizing the transformation logic—such as avoiding unnecessary shuffles or using broadcast joins—reduces peak memory consumption. On the AWS Certified Data Engineer Associate DEA-C01 exam, this question tests your understanding of Glue resource tuning versus code-level fixes; a common trap is choosing Spark’s coalesce, which reduces partitions but doesn’t add memory. Remember the mnemonic “DWO” for DPUs, Worker type, and Optimization to recall the three correct levers for fixing Glue out-of-memory errors.
⚠ Common exam trap
Many exam-takers confuse reducing data volume (S3 Select) with increasing memory capacity, or mistakenly believe coalescing partitions always reduces memory usage, when in fact it can concentrate data and exacerbate OOM errors.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Increase the number of DPUs allocated to the Glue job
Increasing the number of DPUs allocated to the Glue job provides more memory and compute resources for the Spark executors, directly addressing out-of-memory errors by allowing larger datasets to be processed without exceeding heap limits. This is a standard scaling approach for memory-intensive ETL workloads in AWS Glue.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Increase the number of DPUs allocated to the Glue job
Why this is correct
More DPUs provide more memory and compute resources.
- ✗
Use S3 Select to filter data before reading into the Glue job
Why it's wrong here
S3 Select pushes down filtering but does not directly resolve out-of-memory errors.
- ✗
Use Spark's 'coalesce' function to reduce the number of partitions
Why it's wrong here
Coalesce may not free memory; it can even cause out-of-memory if data skew exists.
- ✓
Optimize the transformation logic to use less memory, for example by filtering early
Why this is correct
Reducing data volume early reduces memory pressure.
- ✓
Use a larger worker type, such as G.2X
Why this is correct
Larger worker types have more memory per worker.
Visual reference
Quick reference
AWS S3 Storage Class Comparison
| Storage Class | Min Duration | Retrieval | Use Case |
|---|---|---|---|
| S3 Standard | None | Immediate | Frequently accessed data |
| S3 Standard-IA | 30 days | Immediate | Infrequent access, rapid retrieval |
| S3 One Zone-IA | 30 days | Immediate | Non-critical infrequent data |
| S3 Intelligent-Tiering | None | Immediate–hours | Unknown or changing access patterns |
| S3 Glacier Instant | 90 days | Milliseconds | Archive with instant retrieval |
| S3 Glacier Flexible | 90 days | Minutes–hours | Archive, flexible retrieval |
| S3 Glacier Deep Archive | 180 days | Hours | Long-term compliance archive |
Go deeper
Related to this question
About these practice questions
This DEA-C01 question is part of Courseiva's 1,321-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
2 more ways this is tested on DEA-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A company uses AWS Glue to run ETL jobs that process data from Amazon S3 and load into Amazon Redshift. The jobs have recently started failing with 'Out of Memory' errors. The data volume has increased 3x in the past month. Which is the MOST effective solution to resolve this issue without redesigning the job?
hard- A.Use Amazon Athena instead of Glue for the transformation.
- ✓ B.Increase the number of Glue workers (DPUs) for the job.
- C.Rewrite the job to use Spark SQL instead of PySpark.
- D.Increase the number of partitions in the input S3 data.
Why B: Glue ETL jobs run on a Spark cluster sized by the number of DPUs (workers). Out-of-memory errors during a 3x data volume increase indicate the existing worker count cannot hold the partitions and shuffle data in memory. Increasing the number of Glue workers adds executor memory and parallelism, which is the most direct fix without redesigning the job.
Variation 2. A company is using AWS Glue to run ETL jobs that transform data from Amazon S3 to Amazon Redshift. The jobs are failing intermittently with 'Out of Memory' errors. The team wants to resolve this issue without increasing costs significantly. Which TWO actions should the team take?
medium- A.Increase the Spark memory overhead parameter in the Glue job configuration.
- B.Use DynamicFrame instead of Spark DataFrame for transformations.
- C.Increase the number of workers to maximum allowed.
- D.Switch from a Spark job to a Python shell job.
- ✓ E.Change the worker type from 'G.1x' to 'G.2x' to double memory per worker.
Why E: Option E is correct because switching from G.1x to G.2x workers doubles the memory (and vCPU) per worker while keeping the same worker count, which directly addresses OOM conditions in memory-intensive transformations and is a targeted, cost-controlled change rather than scaling out to maximum workers. Option A is not correct because AWS Glue does not provide a 'Spark memory overhead parameter' in the job configuration; memory overhead is not a user-configurable Glue job setting, so this action cannot be taken as described. Option B is not correct because DynamicFrame vs Spark DataFrame is an API choice for schema handling and does not by itself reduce memory pressure enough to fix OOM errors. Option C is not correct because increasing workers to the maximum allowed raises cost significantly and does not fix per-executor memory limits. Option D is not correct because a Python shell job cannot run distributed Spark ETL at the scale needed for S3-to-Redshift transformations and would not resolve Spark executor OOM issues.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This DEA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DEA-C01 exam.