MLA-C01 Data Preparation for Machine Learning Practice Question

A data engineer notices that an AWS Glue ETL job is failing with an Out of Memory error when processing a large dataset. The dataset is 500 GB in size, and the worker type is G.1X. Which change is MOST likely to resolve the issue?

⚠ Common exam trap

Test-takers frequently assume adding more workers (scaling out) always solves memory issues, but the real bottleneck is per-executor memory, which is only addressed by using a larger worker type (scaling up).

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Use a larger worker type like G.2X

The G.1X worker type provides 16 GB of memory per worker. A 500 GB dataset requires sufficient aggregate memory across workers for processing. Increasing the worker type to G.2X (which doubles memory to 32 GB per worker) increases the memory per executor, allowing each task to handle larger data partitions without running out of memory. This directly addresses the Out of Memory error by providing more heap space for Spark operations.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Partition the input data into smaller files
Why it's wrong here
Partitioning helps parallelism but does not address per-worker memory limits.
✗
Use a Spark DataFrame instead of RDD
Why it's wrong here
Glue already uses DataFrames by default.
✗
Increase the number of workers
Why it's wrong here
Adding more workers does not increase memory per worker; the OOM will persist.
✓
Use a larger worker type like G.2X
Why this is correct
G.2X provides double the memory of G.1X, resolving the OOM.

Visual reference

Go deeper

MLA-C01 Data Preparation for Machine Learning Practice Question

Why each option matters

Option-by-option breakdown

Related to this question