A company uses AWS Glue ETL jobs to transform CSV data from an S3 bucket into Parquet. The jobs often fail with memory errors when processing large datasets. They want to minimize cost and improve reliability. What should they do?
G.1X workers provide more memory and vCPU per worker, reducing OOM errors for memory-intensive transformations.
Why this answer
The G.1X and G.2X worker types provide more memory per worker (16 GB and 32 GB, respectively) compared to the standard G.0X worker (4 GB). By using these worker types and increasing the number of DPUs per worker, you allocate more memory to each task, reducing out-of-memory errors when processing large datasets. This approach also optimizes cost by using fewer, more powerful workers instead of many underpowered ones, improving reliability without unnecessary scaling.
Exam trap
The trap here is that candidates often assume increasing the number of workers (Option D) is the universal fix for performance issues, but in Glue, memory errors are typically caused by insufficient per-worker memory, not a lack of parallelism.
How to eliminate wrong answers
Option B is wrong because Amazon Athena with CTAS queries is a serverless query service, not a data transformation engine; it cannot handle the complex ETL logic (e.g., custom transforms, joins) that Glue jobs perform, and it incurs costs per query scanned, which can be higher for large datasets. Option C is wrong because S3 Batch Operations with AWS Lambda processes files individually, which lacks the distributed, parallel processing capabilities of Glue; it would be slower and more error-prone for large datasets, and Lambda has a 15-minute timeout and limited memory (up to 10 GB), making it unsuitable for memory-intensive transformations. Option D is wrong because simply increasing the number of workers does not address the root cause of memory errors; it adds more parallel tasks but each worker still has the same limited memory (4 GB for G.0X), so tasks can still fail if individual partitions exceed that memory.