A company is designing a real-time analytics pipeline to process streaming data from IoT devices. The solution must be serverless and handle data transformation before storage. Which combination of services is most cost-effective?
Serverless, cost-effective for streaming transformation.
Why this answer
It combines Kinesis Data Streams for real-time ingestion, AWS Lambda for serverless data transformation, and Kinesis Data Firehose to buffer and deliver transformed data to Amazon S3. This architecture is serverless, scales automatically, and is cost-effective as you pay only for data throughput and compute time without provisioning any servers.
Exam trap
The trap here is that candidates often confuse Amazon SQS with Kinesis Data Streams for streaming use cases, but SQS is a pull-based queue with no ordering guarantees across messages and no replay capability, making it unsuitable for real-time analytics pipelines.
How to eliminate wrong answers
Option B is wrong because using Amazon EC2 instances for transformation introduces server management overhead and fixed costs, violating the serverless requirement and increasing cost compared to Lambda. Option C is wrong because Amazon SQS is a message queue designed for decoupling applications, not for real-time streaming analytics; it lacks the ordered, replayable, and high-throughput characteristics needed for IoT streaming data. Option D is wrong because Amazon Kinesis Data Analytics for real-time SQL performs in-stream analytics but does not handle data transformation before storage; it outputs to a destination like Firehose, and alone it cannot transform and store data in S3 without additional services.