DEA-C01 Data Ingestion and Transformation Practice Question

A data engineer is ingesting records from Amazon Kinesis Data Streams into Amazon S3 using AWS Lambda as the consumer. Each stream shard delivers up to 1,000 records per second, and the Lambda function writes each record as an individual small object, causing many tiny S3 files and high PUT costs. The engineer wants fewer, larger objects while keeping near-real-time delivery. What should the engineer do?

⚠ Common exam trap

The trap here is tuning Lambda or stream capacity to fix a file-size problem, when the actual fix is introducing a buffering delivery layer that batches records.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Replace the Lambda consumer with Amazon Data Firehose, which buffers records and delivers batched objects to S3.

The inefficiency comes from writing one S3 object per Kinesis record. Amazon Data Firehose natively buffers records by configurable size and time windows and delivers them as consolidated objects, which reduces file count and PUT costs while still meeting near-real-time needs. Tuning Lambda memory, adding shards, or enabling enhanced fan-out changes throughput and parallelism but not the per-record write pattern.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✓
Replace the Lambda consumer with Amazon Data Firehose, which buffers records and delivers batched objects to S3.
Why this is correct
Amazon Data Firehose reads from Kinesis Data Streams and buffers incoming records by size and time before delivering consolidated objects to S3. This directly produces fewer, larger files and cuts PUT request costs, while its configurable buffer interval preserves near-real-time delivery. It removes the need for custom batching logic in Lambda.
✗
Enable enhanced fan-out on the stream so consumers get dedicated throughput per shard.
Why it's wrong here
Enhanced fan-out gives each consumer dedicated read throughput, reducing contention when multiple consumers read the same stream. It does not batch records or change how the Lambda writes objects to S3. The small-object problem and PUT cost remain unchanged because the write pattern is untouched.
✗
Increase the Lambda function memory so each invocation can write more records per second.
Why it's wrong here
More memory gives the function more CPU and network throughput, but the function still writes each record as its own object unless the code is changed to batch. The number of small objects and PUT costs remain driven by per-record writes. Memory tuning does not address the batching requirement.
✗
Add more shards to the Kinesis data stream to increase parallelism.
Why it's wrong here
Adding shards increases ingest capacity and Lambda concurrency, but more concurrent invocations each writing single-record objects can actually increase the number of tiny files. Resharding does not introduce buffering or batching. The core problem of per-record object creation persists and may worsen.

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

Go deeper

DEA-C01 Data Ingestion and Transformation Practice Question

Why each option matters

Option-by-option breakdown

AWS S3 Storage Class Comparison

Related to this question