A company wants to ingest streaming data from IoT devices into Amazon S3 using Amazon Kinesis Data Firehose. The data must be transformed from JSON to Parquet format before landing in S3. What is the SIMPLEST way to achieve this?
Kinesis Data Firehose offers a native record format conversion feature that transforms incoming JSON to Parquet using a schema from the AWS Glue Data Catalog, with no custom processing required. This directly satisfies the stem's requirement to convert JSON to Parquet before landing in S3, and remains the simplest option since Firehose handles the transformation itself.
Why this answer
Amazon Kinesis Data Firehose has a built-in Parquet conversion feature that uses an AWS Glue schema to convert incoming JSON data to Parquet format. This is the simplest approach because it requires no custom code or additional services; you only need to provide a schema and enable the conversion in the Firehose delivery stream configuration. Option B (using Lambda) is more complex, as it requires writing and maintaining a custom transformation function.
Exam trap
Candidates often overlook the native Parquet conversion capability in Kinesis Data Firehose and assume a Lambda function is required. The built-in conversion using an AWS Glue schema is actually simpler and supported directly.
How to eliminate wrong answers
Option A is wrong because Kinesis Data Firehose does not have a built-in Parquet converter; it can deliver data in Parquet format only if the input data is already in a format that can be converted (e.g., via a Lambda transformation or by using a schema from AWS Glue), but there is no native 'Parquet converter' toggle. Option C is wrong because it introduces unnecessary complexity and latency by storing JSON in S3 first and then running a nightly Glue job, which is not the simplest approach and does not meet the requirement for real-time transformation before landing. Option D is wrong because Kinesis Data Analytics is designed for real-time analytics and stream processing, not for format conversion; it would add unnecessary overhead and complexity compared to using Firehose's built-in Lambda transformation.