A data engineer is troubleshooting a Kinesis Data Analytics application that processes streaming data. The application is falling behind and has a high 'MillisBehindLatest' metric. The application uses a parallelism of 2. The source stream has 4 shards. What is the MOST likely cause and solution?
Trap 1: The application is using a JSON format; switch to Avro.
Data format does not affect processing speed significantly.
Trap 2: The source stream has too many shards; decrease to 2.
Reducing shards decreases throughput capacity, making the problem worse.
Trap 3: The output destination is slow; change to a faster sink.
The high MillisBehindLatest indicates the application is not keeping up with ingestion, not the sink.
- A
The application is using a JSON format; switch to Avro.
Why wrong: Data format does not affect processing speed significantly.
- B
The source stream has too many shards; decrease to 2.
Why wrong: Reducing shards decreases throughput capacity, making the problem worse.
- C
The application parallelism is too low; increase it to 4.
With 4 shards, parallelism should be at least 4 to process all shards concurrently.
- D
The output destination is slow; change to a faster sink.
Why wrong: The high MillisBehindLatest indicates the application is not keeping up with ingestion, not the sink.