When to Use Kinesis Data Analytics for Flink for Stateful Streaming
A data engineer is designing a streaming pipeline that ingests data from an Amazon Kinesis Data Stream (with 5 shards) into Amazon S3. The data must be transformed using a complex stateful operation that cannot be done in a Lambda function (limited to 15 minutes). The engineer needs a solution that can maintain state across multiple records. Which service should be used?
Quick Answer
The answer is Amazon Kinesis Data Analytics for Apache Flink. This service is the correct choice because it is purpose-built for stateful processing, maintaining state across multiple records through checkpointing and fault-tolerant snapshots, which is essential when a complex transformation exceeds the 15-minute Lambda timeout. On the AWS Certified Data Engineer Associate DEA-C01 exam, this question tests your ability to distinguish between stateless and stateful streaming services, with a common trap being Amazon Kinesis Data Firehose, which is stateless and cannot track record history. The exam expects you to recognize that Kinesis Data Analytics for Flink integrates directly with Kinesis Data Streams and handles long-running, stateful logic without requiring cluster management like Amazon EMR. A helpful memory tip is to associate "Flink" with "stateful link" — if your pipeline needs to remember past data, think Flink.
⚠ Common exam trap
Candidates often confuse AWS Glue streaming ETL (which is Spark-based and better for batch-oriented transformations) with a true stateful streaming engine, or they assume Kinesis Data Firehose can handle stateful logic via Lambda, not realizing Lambda's stateless nature and timeout limit.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Amazon Kinesis Data Analytics for Apache Flink
Amazon Kinesis Data Analytics for Apache Flink is the correct choice because it supports stateful stream processing with exactly-once semantics, can maintain state across multiple records, and has no 15-minute execution limit like AWS Lambda. It natively integrates with Kinesis Data Streams and can sink transformed data to S3, meeting all requirements for complex stateful operations.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Amazon EMR running Spark Structured Streaming
Why it's wrong here
EMR can do stateful streaming but requires cluster management, increasing operational overhead.
- ✗
Amazon Kinesis Data Firehose with Lambda transformation
Why it's wrong here
Lambda has a 15-minute timeout and is stateless; not suitable for complex stateful operations.
- ✗
AWS Glue streaming ETL job
Why it's wrong here
Glue streaming is based on Spark and has limited stateful support; less mature than Flink.
- ✓
Amazon Kinesis Data Analytics for Apache Flink
Why this is correct
Flink supports stateful stream processing, exactly what is needed.
Quick reference
AWS S3 Storage Class Comparison
| Storage Class | Min Duration | Retrieval | Use Case |
|---|---|---|---|
| S3 Standard | None | Immediate | Frequently accessed data |
| S3 Standard-IA | 30 days | Immediate | Infrequent access, rapid retrieval |
| S3 One Zone-IA | 30 days | Immediate | Non-critical infrequent data |
| S3 Intelligent-Tiering | None | Immediate–hours | Unknown or changing access patterns |
| S3 Glacier Instant | 90 days | Milliseconds | Archive with instant retrieval |
| S3 Glacier Flexible | 90 days | Minutes–hours | Archive, flexible retrieval |
| S3 Glacier Deep Archive | 180 days | Hours | Long-term compliance archive |
Go deeper
Related to this question
About these practice questions
This DEA-C01 question is part of Courseiva's 1,711-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
2 more ways this is tested on DEA-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A data engineering team is designing a data ingestion pipeline for a social media analytics platform. The pipeline must handle up to 100,000 events per second with less than 1 second processing latency. Which TWO services should be used together to meet these requirements?
medium- A.AWS Glue streaming ETL
- B.Amazon Kinesis Data Firehose
- C.Amazon SQS
- ✓ D.Amazon Kinesis Data Analytics for Apache Flink
- ✓ E.Amazon Kinesis Data Streams
Why D: Amazon Kinesis Data Streams (Option E) is designed for real-time data ingestion at scale, supporting up to 1,000 records per second per shard with sub-second latency, making it suitable for 100,000 events per second when provisioned with sufficient shards. Amazon Kinesis Data Analytics for Apache Flink (Option D) can consume data directly from Kinesis Data Streams and perform low-latency stream processing (e.g., aggregations, filtering) with exactly-once semantics, meeting the <1 second processing latency requirement. Together, they form a fully managed, scalable pipeline for high-throughput, low-latency streaming analytics.
Variation 2. A data engineering team is designing a near-real-time data ingestion pipeline for IoT sensor data. The data must be processed and stored in Amazon S3, with transformations applied before storage. The team needs to handle potential duplicates and ensure exactly-once processing semantics. Which TWO AWS services should be used together? (Choose TWO.)
hard- A.Amazon Kinesis Data Firehose
- B.Amazon Simple Queue Service (SQS)
- ✓ C.Amazon Kinesis Data Analytics for Apache Flink
- ✓ D.Amazon Kinesis Data Streams
- E.AWS Database Migration Service (DMS)
Why C: For near-real-time IoT sensor data ingestion with exactly-once processing, Amazon Kinesis Data Streams (option D) provides ordered, durable, and replayable data ingestion. Amazon Kinesis Data Analytics for Apache Flink (option C) consumes from the stream and supports exactly-once semantics through checkpointing and idempotent sinks, enabling duplicate handling. Amazon Kinesis Data Firehose (option A) offers at-least-once delivery only. Amazon SQS (option B) standard queues do not guarantee exactly-once, and FIFO queues cannot handle high-throughput IoT data. AWS DMS (option E) is for database replication, not streaming ingestion.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This DEA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DEA-C01 exam.