DEA-C01 Data Operations and Support Practice Question

A data engineer is using AWS Step Functions to orchestrate an ETL workflow that includes an AWS Glue job, an Amazon EMR step, and an Amazon Redshift stored procedure. The workflow sometimes fails due to transient errors, and the engineer wants to implement a retry strategy that avoids duplicate data processing. Which TWO actions should the engineer take? (Choose two.)

⚠ Common exam trap

The trap here is thinking that simply increasing timeouts or switching to Express workflows will solve transient errors, when the real solution requires retries and idempotency.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Implement idempotency in the ETL tasks so that retries do not produce duplicate data.

To handle transient errors without duplicating data, the engineer should add Retry policies to Task states and ensure ETL tasks are idempotent. Retry policies automatically re-execute failed tasks, while idempotency guarantees that repeated executions do not create duplicate records. Together, they provide robust error handling and data integrity.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✓
Implement idempotency in the ETL tasks so that retries do not produce duplicate data.
Why this is correct
Idempotency ensures that re-executing a task produces the same result without side effects, such as duplicate records. By designing Glue jobs, EMR steps, and Redshift procedures to be idempotent, retries from Step Functions will not cause data duplication. This is essential when combined with retry policies to safely handle transient failures without compromising data integrity.
✗
Configure the Step Functions execution to run in Express mode for faster retries.
Why it's wrong here
Express workflows are designed for high-volume, short-duration executions and do not support certain features like .sync integrations or activities. They also have different retry semantics. Switching to Express mode would not inherently provide retry capabilities or idempotency; it might even break the workflow due to unsupported integrations. This is not a valid solution for handling transient errors in this scenario.
✗
Use a Catch block to redirect to a cleanup state that deletes partially written data before retrying the entire workflow.
Why it's wrong here
While a Catch block can handle errors, deleting partially written data before retrying the entire workflow is risky and may not prevent duplicates. It also adds complexity and could delete data that is still needed. This approach does not directly address transient errors and may cause more issues than it solves, especially if the workflow is not idempotent.
✓
Add a Retry policy on the individual Task states with a backoff rate and max attempts to handle transient failures.
Why this is correct
Configuring a Retry policy on Task states allows Step Functions to automatically retry failed tasks on transient errors. By specifying a backoff rate and maximum attempts, the workflow can recover without manual intervention. This is a standard approach to handle intermittent issues like network timeouts or service throttling, and it helps avoid duplicate processing if retries are idempotent.
✗
Increase the timeout of the entire state machine to allow more time for tasks to complete.
Why it's wrong here
Increasing the overall timeout does not address transient errors; it only allows more time for long-running tasks. Transient errors typically require retries, not longer timeouts. Moreover, a longer timeout could delay failure detection and recovery. This action alone does not prevent duplicate processing and is not a retry strategy.

Go deeper

DEA-C01 Data Operations and Support Practice Question

Why each option matters

Option-by-option breakdown

Related to this question