DEA-C01 Data Operations and Support Practice Question

A data engineer is troubleshooting a data pipeline that uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The engineer notices that the S3 bucket contains many small files (less than 1 MB). This is causing performance issues in downstream processing. What is the BEST way to reduce the number of small files?

⚠ Common exam trap

DEA-C01 often tests the confusion between buffer size and buffer interval, where candidates might think decreasing interval reduces file count, when it actually increases it.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Increase the buffer size to at least 128 MB in the Firehose delivery stream configuration.

Increasing the Firehose buffer size to at least 128 MB allows more data to accumulate before delivery, resulting in fewer, larger files in S3. This directly addresses the small file problem by batching more records per delivery.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✓
Increase the buffer size to at least 128 MB in the Firehose delivery stream configuration.
Why this is correct
Firehose buffers incoming records and only delivers once the buffer size or interval is reached. Raising the buffer size to 128 MB or more forces larger, less frequent S3 objects, directly reducing the sub-1 MB file count causing downstream performance issues.
✗
Use an AWS Lambda function to transform the data before delivery.
Why it's wrong here
Lambda transformation alters record content, not how often Firehose flushes to S3, so small files persist. Transformation suits format conversion, enrichment or filtering between source and destination. Reducing file count requires increasing the delivery stream's buffer size or buffer interval so more records aggregate per object.
✗
Change the compression format from GZIP to Snappy.
Why it's wrong here
Snappy versus GZIP changes compression ratio and CPU cost, not delivery frequency; Firehose still writes per buffer flush, so file count is unchanged. Snappy suits latency-sensitive consumers needing faster decompression. Buffer size and interval settings govern how many records accumulate per S3 object.
✗
Decrease the buffer interval in the Firehose delivery stream configuration.
Why it's wrong here
Decreasing the buffer interval makes Firehose flush more often, producing more, smaller files and worsening the problem. Short intervals suit near-real-time delivery where latency matters more than object size. Increasing buffer size and interval lets more records accumulate per S3 object, reducing file count.

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

Go deeper

DEA-C01 Data Operations and Support Practice Question

Why each option matters

Option-by-option breakdown

AWS S3 Storage Class Comparison

Related to this question