Increase DPUs in AWS Glue for Batch ETL
A company uses AWS Glue ETL to process data from Amazon S3 and write results to Amazon Redshift. The job fails with a memory error when processing large files. Which action should the data engineer take to resolve this issue?
Quick Answer
The correct action is to increase the number of DPUs allocated to the AWS Glue job. Each Data Processing Unit (DPU) provides a fixed amount of memory and compute, so adding more DPUs directly increases the total memory available for processing large files, resolving the memory error. This approach is the most straightforward way to improve AWS Glue batch ETL performance by increasing DPUs, as it scales resources horizontally without altering the job logic. On the AWS Certified Data Engineer Associate DEA-C01 exam, this question tests your understanding of Glue’s resource allocation model and common troubleshooting for out-of-memory errors. A frequent trap is choosing to reduce parallelism, which might ease memory per worker but often cripples throughput; instead, remember that DPUs are the dial for both memory and compute. Memory tip: “More DPUs, more juice” — when Glue runs out of room, just add more DPUs.
⚠ Common exam trap
Candidates often confuse memory errors with I/O bottlenecks and incorrectly choose S3 Select (Option D) to reduce data volume, when the real issue is insufficient compute memory for Spark transformations.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Increase the number of DPUs allocated to the Glue job.
Increasing the number of DPUs (Data Processing Units) allocated to the AWS Glue job provides more memory and compute resources, which directly addresses the out-of-memory error when processing large files. Glue jobs run on Apache Spark, and insufficient DPUs can cause executors to run out of memory during shuffle or aggregation operations on large datasets.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Reduce the number of partitions in the Glue job.
Why it's wrong here
Fewer partitions may increase data per worker, worsening memory issues.
- ✓
Increase the number of DPUs allocated to the Glue job.
Why this is correct
More DPUs provide additional memory and compute resources.
- ✗
Switch to a smaller instance type in the Glue job configuration.
Why it's wrong here
Smaller instances have less memory, worsening the problem.
- ✗
Use S3 Select to filter columns before reading into Glue.
Why it's wrong here
S3 Select reduces data volume but not the memory footprint of processing.
Visual reference
Quick reference
AWS S3 Storage Class Comparison
| Storage Class | Min Duration | Retrieval | Use Case |
|---|---|---|---|
| S3 Standard | None | Immediate | Frequently accessed data |
| S3 Standard-IA | 30 days | Immediate | Infrequent access, rapid retrieval |
| S3 One Zone-IA | 30 days | Immediate | Non-critical infrequent data |
| S3 Intelligent-Tiering | None | Immediate–hours | Unknown or changing access patterns |
| S3 Glacier Instant | 90 days | Milliseconds | Archive with instant retrieval |
| S3 Glacier Flexible | 90 days | Minutes–hours | Archive, flexible retrieval |
| S3 Glacier Deep Archive | 180 days | Hours | Long-term compliance archive |
Go deeper
Related to this question
About these practice questions
Courseiva writes every DEA-C01 question from scratch — 1,711 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on DEA-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A company uses AWS Glue ETL to transform data from Amazon RDS for PostgreSQL to Amazon S3. The transformation includes joining several tables and aggregating millions of rows. The job runs successfully but takes over 2 hours. The data engineer wants to reduce runtime. Which action is MOST effective?
hard- A.Enable Auto Scaling for the Glue job.
- B.Use AWS Glue DynamicFrames instead of DataFrames.
- ✓ C.Increase the number of DPUs for the Glue job.
- D.Convert the source data to Parquet format.
Why C: Increasing the number of DPUs (Data Processing Units) in AWS Glue ETL jobs allows more parallel processing of the transformation, which directly reduces runtime for CPU-bound or memory-bound tasks like joining and aggregating millions of rows. Option A: Auto Scaling adjusts DPUs based on workload but does not guarantee maximum performance; it may still be limited by the initial DPU allocation. Option B: DynamicFrames vs DataFrames performance difference is minimal for such operations; this would not significantly reduce runtime. Option D: Converting source data to Parquet applies to data in S3, but the source is Amazon RDS, so this conversion does not help with reading from the database. Therefore, increasing DPUs is the most effective action.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This DEA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DEA-C01 exam.