DEA-C01 Data Ingestion and Transformation Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✓

  Increase the number of G.1X workers or use G.2X workers

  Why this is correct

  Glue workers hold the memory used for shuffles and decompression, so adding G.1X workers or switching to G.2X doubles per-worker memory and vCPU. That relieves the out-of-memory condition caused by large compressed files without editing the job script.

• ✗

  Convert the compressed files to uncompressed format before processing

  Why it's wrong here

  Uncompressed files are larger, so executors read more bytes per task and memory pressure increases rather than easing. Decompression is tempting because compressed input can complicate splittable reads, but the correct remedy is allocating more memory per worker, not expanding the data volume.

• ✗

  Repartition the data to fewer partitions

  Why it's wrong here

  Fewer partitions concentrates more data into each executor, raising per-task memory pressure and worsening the out-of-memory failures. Repartitioning is tempting when skew or small-file overhead dominates, but here the fix is raising worker memory or using a larger worker type without touching code.

• ✗

  Increase the job timeout setting

  Why it's wrong here

  A longer timeout only lets the job run before being killed; the executor still exhausts heap while processing the large compressed files, so the out-of-memory error recurs. Timeout tuning suits jobs terminated prematurely on duration, not memory exhaustion, which requires more worker capacity.

Go deeper

Related to this question