DEA-C01 Data Ingestion and Transformation Practice Question

A data engineer is using AWS Glue to process a large dataset stored in Amazon S3 in Parquet format. The Glue job performs a join between two tables and writes the result back to S3. The engineer notices that the job is running slowly and consuming excessive DPU hours. The job has 10 workers of type G.1X. Which action should the engineer take to improve performance and reduce cost?

⚠ Common exam trap

The trap here is assuming that adding more workers always improves performance, when the real issue may be data skew or lack of partitioning.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Partition the Parquet data by the join key and use broadcast join if one table is small.

The performance issue is likely due to inefficient join operations causing large data shuffles. Partitioning the Parquet data by the join key aligns data with the join operation, reducing shuffling. If one table is small enough to fit in memory, a broadcast join eliminates the shuffle entirely. These optimizations reduce runtime and DPU consumption. Other options either do not address the join bottleneck or could increase cost without solving the problem.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Use the 'ApplyMapping' transformation to rename columns before the join.
Why it's wrong here
'ApplyMapping' is used for schema manipulation, such as renaming or changing data types. It does not affect the performance of a join operation. While schema consistency is important, this transformation does not reduce data shuffling or improve partitioning, so it will not speed up the job or reduce DPU usage.
✗
Enable AWS Glue job bookmarks to avoid reprocessing old data.
Why it's wrong here
Job bookmarks help avoid reprocessing previously processed data in incremental workloads, but they do not improve the performance of a single large join operation. The job is already reading the full dataset, so bookmarks would not reduce the runtime or DPU consumption for this join. This action does not address the performance bottleneck.
✗
Increase the number of workers to 20 and keep the worker type as G.1X.
Why it's wrong here
Simply adding more workers without addressing data partitioning or shuffling may not improve performance and could increase cost. If the join is skewed or requires a large shuffle, adding workers might not help and can lead to inefficient resource utilization. The engineer should first optimize the join and data layout.
✓
Partition the Parquet data by the join key and use broadcast join if one table is small.
Why this is correct
Partitioning the data by the join key can reduce data shuffling during the join, and using a broadcast join for a small table avoids a full shuffle altogether. These optimizations can significantly improve performance and reduce DPU hours. This approach directly addresses the join bottleneck and is a best practice in AWS Glue.

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

Go deeper

DEA-C01 Data Ingestion and Transformation Practice Question

Why each option matters

Option-by-option breakdown

AWS S3 Storage Class Comparison

Related to this question