DEA-C01 Data Operations and Support Practice Question

A data engineer is using AWS Glue job bookmarks to process incremental data from Amazon S3. The job reads from a partitioned S3 path and writes to Amazon Redshift. After a recent run, the engineer notices that some new partitions were not processed. The job bookmark state shows that the job has already processed up to a certain timestamp. What is the most likely reason for the missing partitions?

⚠ Common exam trap

The trap here is assuming that all new data will be processed regardless of timestamp, but bookmarks rely on ordering and can miss late-arriving partitions.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

The new partitions were created with a timestamp earlier than the last processed timestamp.

Glue job bookmarks use timestamps or partition values to determine what data has already been processed. If new partitions have timestamps older than the last processed timestamp, they are considered already processed and skipped. This often happens with late-arriving data. The engineer should either adjust the bookmark to use partition-based tracking or handle late data separately.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✓
The new partitions were created with a timestamp earlier than the last processed timestamp.
Why this is correct
Glue job bookmarks track the last processed timestamp or partition. If new data arrives with an older timestamp (e.g., late-arriving data), it may be considered already processed and skipped. This is a common issue with bookmarks when data is not strictly ordered by time. The engineer should adjust the bookmark logic or handle late data.
✗
The Glue job bookmark was not enabled for the S3 data source.
Why it's wrong here
If the bookmark was not enabled, the job would reprocess all data every run, not skip new partitions. The scenario states that bookmarks are used and show a last processed timestamp, so they are enabled. The issue is that some new partitions are not being picked up due to timestamp ordering.
✗
The Redshift cluster's copy command does not support partitioned data.
Why it's wrong here
Amazon Redshift COPY command supports loading from partitioned data in S3. If there were an issue, the job would fail during the write phase, not selectively skip partitions. The problem is about reading new partitions, not writing to Redshift.
✗
The Glue job's IAM role lacks permissions to read the new partitions.
Why it's wrong here
If the IAM role lacked permissions, the job would fail with an access denied error, not silently skip partitions. The scenario indicates the job ran successfully but missed some partitions, so permissions are likely not the issue. The bookmark state suggests the job completed and updated its state.

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

Go deeper

DEA-C01 Data Operations and Support Practice Question

Why each option matters

Option-by-option breakdown

AWS S3 Storage Class Comparison

Related to this question