A company uses Amazon Kinesis Data Firehose to deliver streaming data to Amazon S3. The data is in JSON format. The delivery stream is configured with a buffer size of 5 MB and a buffer interval of 60 seconds. However, the data engineer notices that S3 objects are being created with sizes much smaller than 5 MB. What is a likely cause?
Firehose flushes a buffer when either the 5 MB size or the 60-second interval elapses, whichever occurs first. At a low incoming data rate, the interval triggers repeatedly before the buffer fills, producing S3 objects far smaller than 5 MB.
Why this answer
Kinesis Data Firehose delivers data to S3 when either the buffer size (5 MB) or the buffer interval (60 seconds) is reached, whichever occurs first. If the incoming data rate is low, the buffer interval will expire before accumulating 5 MB of data, resulting in smaller S3 objects.
Exam trap
The trap here is that candidates may assume the buffer size is a hard minimum that must be reached before delivery, but Firehose uses an 'or' condition between buffer size and buffer interval, so low data rate causes interval-based delivery of small objects.
How to eliminate wrong answers
Option A is wrong because compression reduces the size of data after buffering, but the buffer size limit is based on the uncompressed data; compression does not cause smaller objects to be created before the buffer interval triggers. Option C is wrong because a data transformation Lambda can modify records but does not inherently split records into smaller ones; it processes records as a batch and returns them, and any splitting would be a custom logic not default behavior. Option D is wrong because S3 lifecycle policies manage object transitions or deletions after objects are created; they do not split objects during delivery.