DEA-C01 Data Ingestion and Transformation Practice Question

A data engineer runs an AWS Glue ETL job that reads semi-structured JSON from Amazon S3, flattens nested arrays, and writes Parquet to a partitioned S3 location. Job runs are becoming expensive because Glue reprocesses all historical partitions on every run. The engineer wants subsequent runs to process only newly arrived data. Which approach should the engineer take with the LEAST operational overhead?

⚠ Common exam trap

The trap here is assuming a date filter in the transform prevents historical data from being read, when filtering happens after the source objects are already listed and scanned.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Create an AWS Glue job bookmark and enable it on the S3 data source, then let the job read only new objects on each run.

Glue job bookmarks maintain run state and cause the reader to process only new or changed S3 objects since the last successful run. This eliminates repeated scanning of historical partitions with no custom code, unlike runtime filters or schedule changes, which still read all objects. Moving data to RDS is a much heavier architectural change than needed.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Schedule the job with a shorter interval so each run has less data to process.
Why it's wrong here
Shortening the schedule does not change the fact that each run reads all existing objects. It can increase cost by multiplying job runs and does not introduce any state tracking between executions, so the reprocessing problem remains unchanged.
✗
Move the source data to an Amazon RDS table and use a Glue JDBC connection with a watermark column.
Why it's wrong here
Migrating JSON files from S3 into RDS adds significant operational overhead and changes the architecture unnecessarily. While JDBC watermark columns can enable incremental reads, this scenario is specifically about S3 objects and the migration cost far exceeds enabling a native bookmark.
✓
Create an AWS Glue job bookmark and enable it on the S3 data source, then let the job read only new objects on each run.
Why this is correct
Job bookmarks persist state from prior runs and let the Glue job identify newly added S3 objects, so only incremental data is processed. This is the native, low-overhead mechanism for incremental processing in Glue ETL and directly solves the reprocessing cost problem without custom tracking.
✗
Add a WHERE clause to the DynamicFrame that filters records by the current date at runtime.
Why it's wrong here
Filtering by current date inside the transform still requires Glue to list and read every source object before the filter is applied, so historical partitions are still scanned. It reduces output volume but does not prevent redundant I/O, which is the actual cost driver here.

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

Go deeper

DEA-C01 Data Ingestion and Transformation Practice Question

Why each option matters

Option-by-option breakdown

AWS S3 Storage Class Comparison

Related to this question