DEA-C01 Data Operations and Support Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✗

  Enable the Glue job bookmark to skip previously processed files.

  Why it's wrong here

  Job bookmarks track which source objects have already been processed so incremental runs skip them. This reduces the total volume read over successive runs, but it does nothing to change how a single run performs its shuffle. The memory failure occurs within the shuffle stage of the current run, so bookmarking does not lower the peak memory consumed by a single aggregation execution.

• ✓

  Replace the groupByKey operation with reduceByKey or aggregateByKey to combine values before the shuffle.

  Why this is correct

  groupByKey shuffles all values for a key before aggregation, which can produce enormous intermediate data. reduceByKey and aggregateByKey perform map-side combiners that pre-aggregate values locally before the shuffle, dramatically shrinking the data moved across the network. Less shuffled data means smaller per-executor memory footprints during the aggregation, addressing the root cause of the container kills.

• ✗

  Change the output write format from Parquet to uncompressed JSON.

  Why it's wrong here

  Switching to uncompressed JSON increases output size and write time and removes columnar benefits for downstream consumers. It has no effect on the shuffle stage where the memory failure occurs, because the shuffle happens before the write. This change would likely worsen performance and cost without addressing the executor memory exhaustion during aggregation.

• ✗

  Set the job's max concurrent runs to 1 to avoid overlapping executions.

  Why it's wrong here

  Limiting concurrent runs prevents multiple executions of the same job from competing for resources. That is useful for controlling cost and avoiding state conflicts, but it does not change the memory characteristics of a single run. The container kill happens inside one execution's shuffle stage, so serializing runs leaves the underlying memory pressure unresolved.

• ✓

  Increase the number of workers allocated to the Glue job.

  Why this is correct

  Adding workers increases the number of executors available, which distributes the shuffle workload across more containers. With more executors, each one handles a smaller partition of the grouped data, lowering the peak memory any single container must hold. This directly reduces the likelihood of a single executor being killed for exceeding its memory limit during the aggregation stage.

Go deeper

Related to this question