DEA-C01 Data Ingestion and Transformation Practice Question

A data engineer runs a weekly AWS Glue ETL job that processes data from Amazon DynamoDB to Amazon S3. The job reads the entire table every time, which is slow and expensive. The job needs to process only items that changed since the last run. Which solution should the engineer implement?

⚠ Common exam trap

Many candidates think a GSI on a timestamp (Option C) is sufficient for incremental processing, but it fails to capture updates to existing items that do not change the timestamp, and it still requires a full scan of the index to find new records.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Enable DynamoDB Streams and process change events with AWS Lambda to write to S3

DynamoDB Streams captures item-level changes (inserts, updates, deletes) in near real-time. An AWS Lambda function can process these change events and write only the incremental data to Amazon S3, eliminating the need to scan the entire DynamoDB table. This approach is both cost-effective and efficient for incremental data ingestion.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Use DynamoDB Scan with a LastEvaluatedKey to paginate and store the last scanned key to resume next time
Why it's wrong here
LastEvaluatedKey only resumes a Scan's pagination position; it does not identify which items changed since the previous run, so unchanged records are still read. It is tempting because it reduces per-run read volume, and would suit resuming an interrupted full scan, but not incremental change capture.
✓
Enable DynamoDB Streams and process change events with AWS Lambda to write to S3
Why this is correct
DynamoDB Streams captures item-level changes in near real time, and Lambda consumes those events to write only modified items to Amazon S3. This satisfies the requirement to process only items changed since the last run, eliminating the slow, expensive full-table scans the weekly Glue job currently performs.
✗
Add a Global Secondary Index (GSI) on a timestamp attribute and query only new records
Why it's wrong here
A GSI on a timestamp attribute still requires querying it, and DynamoDB GSIs do not track item modifications unless the attribute is rewritten on every update, so changed items are missed. It is tempting because sparse GSIs suit querying recent records, but the correct approach uses DynamoDB Streams to capture changes.
✗
Use AWS Database Migration Service (DMS) with ongoing replication from DynamoDB to S3
Why it's wrong here
AWS DMS ongoing replication captures continuous CDC events from DynamoDB streams, but the requirement is for a weekly batch job that processes only changed items since the last run, not a continuous replication pipeline. DMS would introduce persistent infrastructure costs and schema mapping overhead for a scheduled, incremental read pattern. It is tempting because DMS is designed for continuous data synchronisation between databases and S3, which would be correct if the goal were real-time replication rather than a periodic batch job.

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

Go deeper

DEA-C01 Data Ingestion and Transformation Practice Question

Why each option matters

Option-by-option breakdown

AWS S3 Storage Class Comparison

Related to this question