DEA-C01 Data Ingestion and Transformation Practice Test 3 — 15 Questions

Question 1

A company ingests IoT sensor data into Kinesis Data Streams. The data is then processed by a Lambda function that aggregates readings and writes to DynamoDB. The Lambda function is experiencing high error rates due to throttling. Which TWO actions would reduce throttling?

Accepted Answer

Increase the batch size in the Lambda event source mapping.. Option B is correct because increasing the batch size in the Lambda event source mapping allows each invocation to process more records from the Kinesis stream, reducing the number of concurrent Lambda invocations and thus lowering the risk of throttling. Option D is correct because switching DynamoDB to on-demand capacity mode eliminates write capacity limits, preventing throttling on the DynamoDB side that can cause Lambda retries and backpressure.

Answer

Increase the number of shards in the Kinesis stream.

Answer

Decrease the batch window in the Lambda event source mapping.

Answer

Increase the Lambda reserved concurrency to 1000.

Question 2

A company uses Amazon RDS for MySQL as a source for AWS DMS to replicate data to S3. The replication task is failing with 'OutOfMemory' errors on the DMS instance. The source table has 10 million rows with large BLOB columns. Which THREE changes would most likely resolve the issue?

Accepted Answer

Set the LOB column settings to 'Limited LOB mode' and specify a max LOB size.. Option A is correct because setting LOB columns to 'Limited LOB mode' with a specified max LOB size prevents DMS from loading entire LOBs into memory. Instead, DMS truncates LOBs to the specified size, reducing memory consumption and avoiding OutOfMemory errors when replicating large BLOB columns from MySQL to S3.

Answer

Disable logging for the DMS task to free memory.

Answer

Enable Full LOB mode to handle LOBs more efficiently.

Question 3

A data engineer is troubleshooting a Kinesis Data Firehose delivery stream that ingests JSON log data from web servers. The stream is configured to transform records with an AWS Lambda function and deliver to an Amazon S3 bucket. Recently, the stream has been failing with 'InvalidData' errors. Which action should the engineer take to resolve the issue?

Accepted Answer

Check the CloudWatch Logs for the Lambda function to identify transformation errors.. The 'InvalidData' error in Kinesis Data Firehose typically indicates that the Lambda function used for data transformation is failing or returning malformed records. By checking the CloudWatch Logs for the Lambda function, the engineer can identify specific transformation errors, such as incorrect JSON parsing, missing fields, or exceptions, which cause Firehose to reject the records. This is the most direct troubleshooting step because Firehose relies on the Lambda function to return valid transformed data in the expected format.

Answer

Verify the S3 bucket policy allows Firehose to write.

Answer

Increase the buffer size and interval in the Firehose delivery stream.

Answer

Change the data format to CSV in the Firehose configuration.

Question 4

A data engineer is setting up an Amazon Kinesis Data Analytics application to process streaming data from a Kinesis data stream named "input-stream". The application uses a reference data source from an S3 bucket. The engineer has attached the IAM policy shown in the exhibit to the application's IAM role. When starting the application, the engineer receives an 'AccessDeniedException' error. Which additional permission is required?

Accepted Answer

s3:GetObject on the S3 bucket containing the reference data. The Kinesis Data Analytics application needs to read reference data from the S3 bucket, which requires the s3:GetObject permission on the bucket and its objects. The error 'AccessDeniedException' indicates the IAM role lacks this specific permission to retrieve the reference data file. Option B correctly adds the missing s3:GetObject action to allow the application to fetch the reference data from S3.

Answer

kinesis:PutRecord on the input stream

Answer

kinesis:CreateStream on the input stream

Answer

kinesis:PutRecords on the input stream

Question 5

A company runs an e-commerce platform that generates clickstream data from millions of users. The data is ingested into Amazon Kinesis Data Streams with a shard count of 10. The data is then consumed by a Kinesis Data Analytics application that runs SQL queries to aggregate metrics in real time. Recently, the application has been falling behind, and the stream's iterator age metric is increasing. The data volume has doubled over the past month. The application currently uses a single Kinesis Data Analytics application with parallelism of 1. Which action should the data engineer take to improve the processing rate and reduce the iterator age without losing data or causing duplicates?

Accepted Answer

Increase the parallelism of the Kinesis Data Analytics application to match the number of shards.. Option D is correct because Kinesis Data Analytics (KDA) processes data from each shard in a stream using one or more parallel operators. With a parallelism of 1, the application uses only a single processing thread, which cannot keep up with the doubled data volume across 10 shards. By increasing parallelism to match the shard count (10), KDA can read from all shards concurrently, distributing the processing load and reducing the iterator age without data loss or duplicates, as KDA manages checkpointing and exactly-once semantics internally.

Answer

Change the Kinesis Data Analytics application to use a Kinesis Data Firehose delivery stream as the source.

Answer

Reduce the retention period of the Kinesis Data Streams to 24 hours.

Answer

Increase the number of shards in the Kinesis Data Streams to 20.

Question 6

A financial services company ingests real-time stock trade data from multiple exchanges into Amazon Kinesis Data Streams. Each trade record is a JSON object containing fields: trade_id, symbol, price, quantity, and timestamp. The data is consumed by an AWS Lambda function that performs data validation and enrichment, then writes the processed records to an Amazon DynamoDB table for low-latency querying. Recently, the Lambda function has been timing out and failing to process all records. The Lambda function is configured with a 5-second timeout and 128 MB memory. The average record size is 2 KB, and the stream receives about 1000 records per second. The Lambda function's concurrency limit is 1000. Which set of actions should the data engineer take to resolve the issue without losing data?

Accepted Answer

Increase the Lambda function timeout to 60 seconds and memory to 1024 MB. Set the batch size to 100 records and enable parallelization factor of 10.. Option A is correct because increasing the Lambda timeout and memory addresses the processing bottleneck, while setting the batch size to 100 and enabling a parallelization factor of 10 allows each shard to process up to 10 concurrent batches, dramatically increasing throughput to handle 1000 records/sec (each shard can process 10 batches of 100 records concurrently, yielding 1000 records/sec per shard if the stream has at least 1 shard). This combination ensures no data loss by keeping up with the ingestion rate without exceeding the Lambda concurrency limit of 1000.

Answer

Increase the number of shards in the Kinesis data stream to 20 and keep the Lambda configuration unchanged.

Answer

Replace the Lambda function with a Kinesis Data Firehose delivery stream that writes directly to DynamoDB using a Lambda transformation.

Answer

Increase the Lambda function timeout to 60 seconds and memory to 1024 MB. Set the batch size to 100 records.

Question 7

A company uses AWS Glue to process streaming data from Amazon Kinesis Data Streams. The data is JSON formatted and includes a timestamp field. The company wants to partition the output in Amazon S3 by date and hour, and ensure exactly-once processing semantics. Which combination of configurations should be used?

Accepted Answer

Enable checkpointing in the AWS Glue streaming job and specify an S3 location for checkpoint data.. Option B is correct because AWS Glue streaming jobs require checkpointing to track the progress of data consumption from Kinesis Data Streams and to ensure exactly-once processing semantics. By enabling checkpointing and specifying an S3 location, Glue periodically saves the state of processed records, allowing it to resume from the last committed offset in case of failures, thus preventing duplicates or data loss.

Answer

Disable checkpointing and use the 'exactly_once' delivery option in Kinesis Data Streams.

Answer

Use Amazon DynamoDB as a checkpoint store by configuring the Glue job with a DynamoDB connection.

Answer

Use Kinesis Client Library (KCL) checkpointing with a DynamoDB table.

Question 8

A data engineer is troubleshooting an AWS Glue ETL job that reads from Amazon S3 and writes to Amazon Redshift. The job runs successfully but writes duplicate rows into Redshift. The source data is static and does not contain duplicates. Which configuration change is most likely to resolve this issue?

Accepted Answer

Modify the job to use the 'postactions' option with a SQL statement that deletes duplicates before final insert.. The job runs successfully but writes duplicate rows because AWS Glue's Spark-based ETL jobs can retry tasks on failure, and when writing to Redshift using the JDBC connector, the default behavior is to append data without deduplication. Using the 'postactions' option with a SQL DELETE statement that removes duplicates before the final INSERT ensures that only unique rows remain, resolving the duplication without altering the source data.

Answer

Enable the 'upsert' feature in the Redshift connection by setting 'update' to true.

Answer

Use partition pruning on the S3 source to reduce the number of files read.

Answer

Increase the number of DPUs (Data Processing Units) allocated to the job.

Question 9

A company is designing a data ingestion pipeline to load CSV files from an SFTP server into Amazon S3. The files are generated hourly and range from 10 MB to 500 MB. Which AWS service should be used to orchestrate the transfer with minimal operational overhead?

Accepted Answer

AWS Transfer Family. AWS Transfer Family is the correct choice because it provides a fully managed, serverless SFTP endpoint that can directly receive files from an SFTP server and automatically store them in Amazon S3. This eliminates the need to manage any compute infrastructure or write custom code for the transfer, minimizing operational overhead for hourly CSV file ingestion.

Answer

AWS Glue

Answer

Amazon AppFlow

Answer

AWS DataSync

Question 10

A company uses Amazon Kinesis Data Firehose to ingest log data from web servers into Amazon S3. The data is in JSON format and each record is approximately 2 KB. The delivery stream is configured to buffer incoming records for 60 seconds or 5 MB, whichever comes first. The company notices that the data in S3 is delayed by up to 5 minutes during peak hours. Which action would most effectively reduce the delivery latency?

Accepted Answer

Decrease the buffer interval to 15 seconds.. The observed delay of up to 5 minutes during peak hours indicates that the buffer size threshold (5 MB) is rarely reached because each record is only ~2 KB, so the delivery stream relies on the buffer interval (60 seconds) to trigger delivery. By decreasing the buffer interval to 15 seconds, Kinesis Data Firehose will push data to S3 more frequently, directly reducing the maximum latency from 60 seconds to 15 seconds per batch, which eliminates the compounding delays caused by queuing during high-throughput periods.

Answer

Increase the buffer size to 10 MB to allow more records per delivery.

Answer

Enable compression (GZIP) on the delivery stream.

Answer

Enable data transformation with AWS Lambda to convert JSON to Parquet.

Question 11

A company is ingesting real-time clickstream data into Amazon S3 using Amazon Kinesis Data Firehose. The data is semi-structured and the company wants to transform the data into Parquet format and partition it by year, month, day, and hour. Which TWO steps should be taken to achieve this? (Choose TWO.)

Accepted Answer

Enable dynamic partitioning in Kinesis Data Firehose and specify the partition keys as year, month, day, hour extracted from the data.. Option B is correct because Kinesis Data Firehose's dynamic partitioning feature allows you to specify partition keys (year, month, day, hour) extracted from the incoming data, and Firehose will automatically create the corresponding S3 prefix structure (e.g., year=2024/month=01/day=15/hour=10/) during delivery. Option D is correct because to convert semi-structured data to Parquet format, you can attach an AWS Lambda function as a data transformation to Firehose, which converts each record to Parquet before delivery to S3.

Answer

Set up an Amazon S3 event notification to trigger an AWS Lambda function that partitions the data after delivery.

Answer

Use an AWS Glue Crawler to infer the schema and automatically partition the data in S3.

Answer

Configure Kinesis Data Firehose to convert the data to Parquet format using a schema from the AWS Glue Data Catalog.

Question 12

Arrange the steps to create an AWS Glue job that transforms data from Amazon S3 to Amazon Redshift in the correct order.

Question 13

Arrange the steps to implement a data lake on Amazon S3 with AWS Lake Formation.

Question 14

Order the steps to query data in Amazon Redshift Spectrum from an external table in Athena.

Question 15

Order the steps to troubleshoot a failed AWS Glue job that reads from JDBC and writes to S3.