1
Data Ingestion and Transformation
hard
A data engineer is using AWS Glue to read data from an Amazon Kinesis Data Stream. The Glue job is configured to process the stream in micro-batches. The engineer notices that the job is not processing all records and sometimes skips data. The Kinesis stream has multiple shards, and the Glue job is using the 'kinesis' connection type. What is the most likely cause of the missing records?