DEA-C01 Data Ingestion and Transformation Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✓

  Partition the source Parquet data in Amazon S3 by commonly filtered columns and use partition pruning in the Glue job.

  Why this is correct

  Partitioning the source data by columns used in filters allows Glue to read only relevant partitions, reducing I/O and the amount of data shuffled during joins. This can dramatically improve performance for large datasets. The engineer should ensure the Glue job uses predicate pushdown and that the partition columns are used in the query. This is a best practice for optimizing Glue ETL on S3.

• ✗

  Enable job bookmarks to avoid reprocessing previously processed data.

  Why it's wrong here

  Job bookmarks help avoid reprocessing old data on subsequent runs, which can reduce runtime for incremental loads. However, for a single job run that processes a large dataset, bookmarks do not speed up the transformation itself. They are useful for incremental processing but do not address the performance of joins or writes within a run. Thus, they are not a primary performance improvement for the described scenario.

• ✓

  Increase the number of AWS Glue DPUs allocated to the job to add more Spark executors.

  Why this is correct

  Increasing DPUs provides more compute resources, which can parallelize reads, joins, and writes. For large Parquet datasets and joins, additional executors reduce the time spent on shuffles and processing. This is a direct way to improve performance when the job is resource-bound, though it increases cost. The engineer should monitor CloudWatch metrics to ensure the job is not bottlenecked elsewhere.

• ✗

  Convert the Parquet files to CSV to reduce storage size and speed up reading.

  Why it's wrong here

  CSV is less efficient than Parquet for analytics; Parquet is columnar, compressed, and supports predicate pushdown. Converting to CSV would increase storage size and slow down reads, not improve performance. This would be a step backward and is not a valid optimization for a Glue job reading large datasets.

• ✗

  Use the 'Relationalize' transform to flatten the Parquet data before joining.

  Why it's wrong here

  Relationalize is for flattening nested data, not for improving join performance on already flat Parquet files. Applying it would add unnecessary processing and likely increase runtime. The scenario describes large Parquet files that are already suitable for joins; Relationalize is not relevant and would not help performance.

Go deeper

Related to this question