A SysOps administrator manages a fleet of EC2 instances that run a batch processing job. The job runs every hour and takes about 45 minutes to complete. The administrator wants to be notified if any job takes longer than 1 hour. Currently, the administrator uses CloudWatch Logs to capture job start and end times from application logs. The job writes a log message at start with 'JOB_START' and at end with 'JOB_END'. The administrator wants to create a metric filter that counts jobs that exceed 1 hour. However, the administrator is unsure how to achieve this with CloudWatch Logs. What should the administrator do?
CloudWatch Events (EventBridge) can deliver CloudWatch Log events to a Lambda function in near real-time via a subscription filter, enabling event-driven processing. The Lambda function can parse the JOB_START and JOB_END entries, correlate them by job ID, calculate the duration, and publish a custom metric or trigger an alarm. This serverless architecture avoids polling and reacts immediately to each logged job, making it the recommended pattern.
Why this answer
CloudWatch Events (now part of Amazon EventBridge) can capture log events in real-time and trigger a Lambda function. The Lambda function can then compute job duration by correlating JOB_START and JOB_END events (e.g., using a DynamoDB table to store start times) and publish a custom metric or trigger an alarm if duration exceeds 1 hour. This approach handles the per-job correlation that metric filters cannot achieve.
Exam trap
Candidates often think metric filters can compute duration by extracting timestamps from JOB_START and JOB_END, but metric filters operate on individual log events and cannot correlate two events for the same job. The correct solution uses CloudWatch Events with Lambda for stateful computation.
How to eliminate wrong answers
Option A is wrong because CloudWatch Logs Insights is a query-based analysis tool for ad-hoc or scheduled queries, but it cannot directly trigger alarms or continuously monitor for durations exceeding 1 hour without custom scripting and additional services. Option B is wrong because CloudWatch Events (now Amazon EventBridge) can capture log events and trigger a Lambda function, but this approach adds unnecessary complexity and cost compared to a native metric filter, and it requires custom code to compute duration and publish metrics. Option D is wrong because S3 is not involved in the described workflow; the logs are in CloudWatch Logs, not S3, and using S3 triggers would require exporting logs to S3 first, adding latency and complexity.