DEA-C01 Data Operations and Support Practice Question

A data engineer is using Amazon Kinesis Data Firehose to deliver streaming data to an Amazon S3 bucket. The engineer notices that some records are being delivered to S3 with a delay of several minutes, and sometimes records are missing. The Firehose stream is configured with a buffer size of 5 MB and a buffer interval of 300 seconds. The engineer wants to reduce latency and ensure all records are delivered. Which action should the engineer take?

⚠ Common exam trap

The trap here is thinking that increasing buffer size improves latency, when actually it increases the time data waits to be flushed.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Reduce the buffer interval to 60 seconds and enable Amazon CloudWatch Logs for error logging.

The buffer interval determines how long Firehose waits before delivering data. A 300-second interval means records can be delayed up to 5 minutes. Reducing it to 60 seconds lowers latency. Missing records often indicate delivery errors, which can be diagnosed by enabling CloudWatch Logs for the Firehose stream. Firehose logs errors related to S3 delivery, Lambda processing, and format conversion, helping identify and fix issues.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✓
Reduce the buffer interval to 60 seconds and enable Amazon CloudWatch Logs for error logging.
Why this is correct
Reducing the buffer interval lowers the maximum time records wait before delivery, decreasing latency. Enabling CloudWatch Logs allows you to capture delivery errors and troubleshoot missing records. Firehose can log errors to CloudWatch, which helps identify why records are not delivered, such as permission issues or malformed records.
✗
Increase the buffer size to 128 MB to allow more data to be batched.
Why it's wrong here
Increasing the buffer size would allow more data to accumulate before delivery, which could increase latency, not reduce it. It might improve throughput but does not address the missing records or the delay. Larger buffers mean records wait longer to be delivered, worsening the latency issue.
✗
Enable Amazon S3 versioning on the destination bucket.
Why it's wrong here
S3 versioning protects against accidental overwrites or deletions, but it does not affect Firehose delivery latency or missing records. Firehose delivers data as objects; versioning would not prevent records from being missing due to delivery failures. It is unrelated to the problem.
✗
Configure the Firehose stream to use AWS Lambda for data transformation.
Why it's wrong here
Lambda transformation can modify records but does not inherently reduce latency or prevent missing records. In fact, if the Lambda function fails, records might be delivered to an error prefix or dropped, potentially causing more missing data. It does not address the root cause of delayed or missing deliveries.

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

Go deeper

DEA-C01 Data Operations and Support Practice Question

Why each option matters

Option-by-option breakdown

AWS S3 Storage Class Comparison

Related to this question