DEA-C01 Data Operations and Support Practice Question

A data engineer is responsible for monitoring an AWS Glue ETL job that runs daily. The job reads data from an Amazon S3 bucket and writes to an Amazon Redshift table. The engineer wants to receive an alert if the job fails or if it takes longer than expected to complete. Which AWS service should the engineer use to set up these alerts with the LEAST operational overhead?

⚠ Common exam trap

The trap here is overcomplicating the solution with Lambda and SES, when CloudWatch Alarms with SNS directly provide the needed alerting.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Amazon CloudWatch Alarms based on AWS Glue job metrics, with Amazon SNS notifications.

Amazon CloudWatch Alarms can monitor AWS Glue job metrics and trigger Amazon SNS notifications when thresholds are exceeded. This setup requires no custom code and provides timely alerts on job failures or prolonged execution, making it the most efficient solution with minimal operational overhead.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Amazon CloudWatch Events (EventBridge) to trigger an AWS Lambda function that sends an email via Amazon SES.
Why it's wrong here
This approach requires writing and maintaining a Lambda function and configuring SES, which adds operational overhead. While it can work, it is not the simplest solution. CloudWatch Events can directly trigger an SNS topic, eliminating the need for custom code. The engineer should aim for a serverless, low-maintenance solution.
✗
AWS Glue job bookmarks to track job progress and send notifications on failure.
Why it's wrong here
Job bookmarks are used to track processed data to avoid reprocessing, not for sending alerts. They do not provide notification capabilities. The engineer would still need to monitor job status separately. Therefore, this does not meet the requirement for alerting on failures or long runtimes.
✗
AWS CloudTrail to log AWS Glue API calls and analyze logs for failures.
Why it's wrong here
CloudTrail records API activity, but it does not provide real-time alerting on job failures or durations. Analyzing logs manually or with additional tools would be required, which is not efficient for timely alerts. CloudTrail is more suited for auditing and compliance, not for operational monitoring of job status.
✓
Amazon CloudWatch Alarms based on AWS Glue job metrics, with Amazon SNS notifications.
Why this is correct
CloudWatch Alarms can monitor AWS Glue job metrics such as glue.driver.aggregate.elapsedTime and glue.driver.aggregate.numFailedTasks. When a threshold is breached, the alarm can publish to an SNS topic, sending notifications. This requires minimal setup and no custom code, making it the least operational overhead solution for alerting on job failures and durations.

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

Go deeper

DEA-C01 Data Operations and Support Practice Question

Why each option matters

Option-by-option breakdown

AWS S3 Storage Class Comparison

Related to this question