DEA-C01 Data Ingestion and Transformation Practice Test 27 — 15 Questions

Question 1

A company uses AWS Lambda to process messages from an Amazon SQS queue. The messages contain JSON payloads that need to be transformed and written to an Amazon DynamoDB table. Recently, the Lambda function has been timing out and messages are being sent to the dead-letter queue (DLQ). What is the BEST way to troubleshoot and resolve this issue?

Accepted Answer

Increase the Lambda function timeout and monitor DynamoDB write capacity to ensure it is not throttling.. Option B is correct because increasing the Lambda timeout and checking DynamoDB write capacity are the first steps to address timeouts. Option A is wrong because increasing SQS visibility timeout alone does not help if Lambda cannot process within the timeout. Option C is wrong because converting to a FIFO queue does not solve timeout issues. Option D is wrong because using a standard queue does not affect processing time.

Answer

Use a standard SQS queue instead of a DLQ to reprocess failed messages automatically.

Answer

Switch the SQS queue to a FIFO queue to ensure exactly-once processing.

Answer

Increase the visibility timeout of the SQS queue to 30 minutes.

Question 2

A data pipeline uses Amazon Kinesis Data Firehose to ingest log data from web servers and deliver it to Amazon S3. The data is then transformed by an AWS Glue job before being loaded into Amazon Redshift. The pipeline must handle a sudden spike in log volume without data loss. Which configuration change is MOST appropriate?

Accepted Answer

Configure Kinesis Data Firehose to back up all data to S3 in case of delivery failures.. Option C is correct because enabling S3 backup for failed records in Firehose ensures that if the delivery fails after retries, the data is stored in S3 and can be reprocessed later, preventing data loss. Option A is wrong because increasing S3 bucket size doesn't help with data loss. Option B is wrong because increasing Glue job timeout does not prevent data loss during ingestion. Option D is wrong because increasing Redshift concurrency does not affect the ingestion pipeline.

Answer

Increase the AWS Glue job timeout and allocate more DPUs.

Answer

Increase the number of nodes in the Redshift cluster to handle higher load.

Answer

Increase the S3 bucket size limit and enable versioning.

Question 3

A company wants to ingest streaming data from IoT devices into Amazon S3 using Amazon Kinesis Data Firehose. The data must be transformed from JSON to Parquet format before landing in S3. What is the SIMPLEST way to achieve this?

Accepted Answer

Use an AWS Lambda function as a data transformation in Kinesis Data Firehose to convert JSON to Parquet.. Option D is correct because Kinesis Data Firehose can use an AWS Lambda function for data transformation, and converting to Parquet can be done via a Lambda function. Option A is wrong because Firehose can deliver to S3 directly. Option B is wrong because Firehose can transform without a separate Glue job. Option C is wrong because Firehose does not natively support Parquet conversion without a transformation.

Answer

Configure Kinesis Data Firehose with a built-in Parquet converter.

Answer

Use Kinesis Data Firehose to deliver data directly to S3 in JSON format and run a nightly Glue job to convert to Parquet.

Answer

Use Kinesis Data Analytics to convert the data to Parquet before sending to Firehose.

Question 4

A data engineer is designing a pipeline to ingest change data capture (CDC) events from an Amazon RDS for PostgreSQL database into Amazon S3. The CDC events are captured using AWS DMS. The data must be available for querying within 5 minutes of the change. Which approach meets these requirements?

Accepted Answer

Use AWS DMS to replicate data directly to S3 in near real-time.. Option A is correct because DMS can stream CDC data to S3 in near real-time, meeting the 5-minute SLA. Option B is wrong because a Lambda function triggered by S3 events is not needed for the initial ingestion. Option C is wrong because exporting to S3 and then using Glue adds latency. Option D is wrong because Redshift is a separate service and adds complexity.

Answer

Export the database to S3 using pg_dump and then use AWS Glue to load into S3 in Parquet format.

Answer

Use AWS DMS to replicate data to Amazon Redshift, then unload to S3.

Answer

Use AWS DMS to replicate data to an SQS queue, then process with Lambda to write to S3.

Question 5

A company uses Amazon Kinesis Data Analytics for real-time anomaly detection on clickstream data. The application uses a sliding window of 1 minute. The data engineer notices that the application is producing incorrect results because late-arriving records are not being handled properly. What should the data engineer do to ensure late records are included in the window calculations?

Accepted Answer

Increase the watermark delay in the Kinesis Data Analytics application to allow more time for late records.. Option B is correct because Kinesis Data Analytics allows configuring a higher watermark delay, which tells the application to wait longer for late-arriving records. Option A is wrong because increasing the window size changes the semantics. Option C is wrong because modifying the stream retention does not affect the application's window. Option D is wrong because ordering is not the issue; it's about waiting for late data.

Answer

Use a Kinesis Data Firehose to buffer the data and then send to Kinesis Data Analytics.

Answer

Increase the window size from 1 minute to 2 minutes.

Answer

Increase the retention period of the Kinesis stream to 7 days.

Question 6

A company needs to ingest data from a self-managed Apache Kafka cluster running on EC2 into Amazon S3. The data must be delivered in near real-time. Which AWS service is BEST suited for this task?

Accepted Answer

Use an AWS Glue streaming ETL job that reads from the Kafka cluster and writes to S3.. Option C is correct because Amazon MSK (Managed Streaming for Kafka) is a managed Kafka service, but the question asks for ingestion from a self-managed cluster. Using MirrorMaker or a Kafka Connect S3 connector can replicate data to S3. However, the simplest managed solution is to use Amazon Kinesis Data Firehose with a Kafka connector (if using Firehose's HTTP endpoint), but the best answer is to use MSK as a target? Wait, the question is about ingesting into S3. Actually, the best practice is to use a Kafka Connect S3 sink connector. The closest AWS service that can help is AWS Glue? No. Let's reconsider: The correct answer is to use a Kafka Connect S3 sink connector running on an EC2 instance, but that's not an AWS service. Among AWS services, the best is Amazon Kinesis Data Firehose with a custom producer that reads from Kafka. But the simplest is to use Amazon MSK as a target? No, the question says from self-managed Kafka to S3. The most appropriate answer is to use an AWS Lambda function that consumes from Kafka and writes to S3, but that's not the best. Actually, the best answer is to use a Kafka Connect S3 sink connector deployed on an EC2 instance. But since the options must be AWS services, the correct one is Amazon Kinesis Data Firehose with a custom application that writes to Firehose. However, among the given options, D is the most direct: Use a Kafka Connect S3 sink connector (which is an open-source connector that can be run on EC2). But the question asks for an AWS service. The answer should be: Use Amazon Kinesis Data Firehose with a custom producer. But that's not listed. Let's design the options: A) AWS Glue, B) Amazon Kinesis Data Streams, C) Amazon MSK, D) Amazon S3 Transfer Acceleration. None are perfect. The best is to use a Kafka Connect S3 sink connector on EC2, but that's not a service. So I'll choose C) Use Amazon MSK as an intermediary? That doesn't make sense. Actually, the correct answer is to use a Kafka Connect S3 sink connector, but since it's not an AWS service, the next best is to use AWS Glue with a Kafka source? Glue can read from Kafka. So Option A: Use AWS Glue ETL job with a Kafka source and write to S3. That is plausible. So I'll set A as correct. Explanation: Glue can connect to Kafka and write to S3 in near real-time using streaming ETL. Option B: Kinesis Data Streams would require a separate connector. Option C: MSK is a managed Kafka, not a solution for ingesting from self-managed Kafka to S3. Option D: S3 Transfer Acceleration is for speeding up uploads, not for ingestion from Kafka.

Answer

Use Amazon MSK to replicate the Kafka cluster and then use a connector to S3.

Answer

Use Amazon S3 Transfer Acceleration to speed up the transfer from Kafka brokers to S3.

Answer

Use Amazon Kinesis Data Streams as an intermediary to buffer data before writing to S3.

Question 7

A data engineer is designing a pipeline to ingest daily CSV files from an SFTP server into Amazon S3. The files are large (up to 10 GB) and must be encrypted in transit. The pipeline should be fully managed and serverless where possible. Which TWO services should be used together to achieve this? (Choose TWO.)

Accepted Answer

AWS Glue. Options B and D are correct. AWS Transfer Family provides a fully managed SFTP service that can transfer files directly to S3 with encryption in transit. AWS Glue can then be used to process the CSV files in S3. Option A is wrong because Lambda has a 15-minute timeout and may not handle large files. Option C is wrong because Kinesis is for streaming data, not batch file transfers. Option E is wrong because Athena is a query service, not a transfer service.

Answer

Amazon Kinesis Data Firehose

Answer

AWS Lambda

Answer

Amazon Athena

Question 8

A company is ingesting IoT sensor data into Amazon Kinesis Data Streams. Each sensor sends a JSON payload every second. The data must be transformed and aggregated in real-time before being stored in Amazon DynamoDB. Which THREE services should be used together in the pipeline? (Choose THREE.)

Accepted Answer

AWS Lambda. Options A, C, and D are correct. Kinesis Data Streams ingests the data. Kinesis Data Analytics performs real-time transformations and aggregations. Lambda can be used to write the aggregated results to DynamoDB. Option B is wrong because Kinesis Data Firehose is for delivery to S3 or Redshift, not DynamoDB. Option E is wrong because S3 is not needed for this pipeline.

Answer

Amazon S3

Answer

Amazon Kinesis Data Firehose

Question 9

A company wants to use AWS Glue to transform data stored in Amazon S3. The data is partitioned by date and includes both CSV and Parquet files. The transformation should be optimized for cost and performance. Which THREE actions should the data engineer take? (Choose THREE.)

Accepted Answer

Use partition pruning by filtering on the date column in the ETL script.. Options A, C, and D are correct. Partition pruning reduces the amount of data scanned. Using Parquet improves performance and compression. Using a job bookmark prevents reprocessing of old data. Option B is wrong because increasing the number of DPUs may increase cost without optimization. Option E is wrong because a crawler is not needed if the schema is known.

Answer

Run a crawler to update the schema before each job run.

Answer

Increase the number of DPUs to the maximum allowed.

Question 10

Refer to the exhibit. An IAM policy is attached to an EC2 instance role that runs a data ingestion application. The application reads files from an S3 bucket 'data-lake-primary' and sends records to a Kinesis stream named 'clickstream'. The application is failing with an 'AccessDenied' error when trying to read from S3. What is the MOST likely cause?

Accepted Answer

The policy does not include the s3:ListBucket permission.. Option C is correct. The policy allows s3:GetObject but the resource is specified with a trailing slash and wildcard (/*). If the application is trying to read an object in a subfolder, the policy may still work, but the error could be due to missing s3:ListBucket permission for the bucket itself when the application lists objects. The most common cause is that the application is trying to list the bucket (s3:ListBucket) which is not allowed. Option A is wrong because the stream name is correct. Option B is wrong because the actions are allowed. Option D is wrong because the policy is attached to the role.

Answer

The actions are specified incorrectly; they should be s3:GetObject and s3:PutObject only.

Answer

The policy is not attached to the EC2 instance role.

Answer

The Kinesis stream name is incorrect.

Question 11

Refer to the exhibit. A data engineer runs the describe-stream command and sees this output. The application is writing records to the stream but is experiencing high write latency. The average record size is 50 KB, and the write rate is 1500 records per second. What is the MOST likely cause of the latency?

Accepted Answer

The stream does not have enough shards to handle the write throughput.. Option A is correct. Each shard can ingest up to 1 MB/s or 1000 records/s (for 1 KB records). With 2 shards, the total capacity is 2000 records/s, but the record size is 50 KB, so the throughput in MB/s is 1500 * 50 KB = 75 MB/s, far exceeding the 2 MB/s total write capacity. The shards are overloaded. Option B is wrong because provisioned throughput is not relevant for Kinesis. Option C is wrong because the application does not need to be in the same region for performance. Option D is wrong because there is no mention of throttling from downstream.

Answer

The application is running in a different AWS region.

Answer

The application is exceeding the DynamoDB provisioned throughput.

Answer

The Kinesis stream is throttling the application because of a hot shard.

Question 12

Refer to the exhibit. A data engineer runs this AWS Glue Data Catalog DDL statement to create a table. The CSV files in 's3://my-bucket/sales/' use a pipe delimiter (|) instead of a comma. What change is needed to correctly read the data?

Accepted Answer

Change the 'field.delim' property to '|'.. Option A is correct. The SERDEPROPERTIES specify 'field.delim' = ',' but the files use pipe. Changing the delimiter to '|' will allow correct parsing. Option B is wrong because partition projection is not needed. Option C is wrong because the table already exists. Option D is wrong because the location is correct.

Answer

Change the LOCATION to read from a subfolder.

Answer

Add a partition projection configuration.

Answer

Run a crawler to detect the schema automatically.

Question 13

A company wants to ingest streaming data from thousands of IoT devices into AWS for real-time analytics. The data volume is variable and can spike unpredictably. The solution must be serverless and minimize operational overhead. Which AWS service should be used for ingestion?

Accepted Answer

Use Amazon Kinesis Data Streams to ingest and process data in real time.. Option B is correct because Amazon Kinesis Data Streams is a serverless streaming data service that can handle variable and high-throughput data from many sources, making it ideal for IoT data ingestion. Option A (Amazon SQS) is for message queuing, not real-time streaming analytics. Option C (Amazon Kinesis Data Firehose) is for loading streaming data into data stores, but it does not provide the same real-time processing capabilities as Kinesis Data Streams. Option D (AWS IoT Core) is a managed cloud service that lets connected devices easily and securely interact with cloud applications and other devices, but it is not primarily a streaming ingestion service for analytics.

Answer

Use Amazon Kinesis Data Firehose to load streaming data directly into Amazon S3.

Answer

Use Amazon SQS to queue messages and process them in batches.

Answer

Use AWS IoT Core to ingest data and route it to Amazon DynamoDB.

Question 14

A data engineer needs to transfer 50 TB of historical data from an on-premises HDFS cluster to Amazon S3. The on-premises network has a 1 Gbps link to AWS. The transfer must complete within 5 days. Which solution is MOST cost-effective and meets the requirements?

Accepted Answer

Use AWS Snowball Edge to physically transfer the data.. Option C is correct because AWS Snowball Edge is a physical device that can transfer large amounts of data faster than a network link, especially with a 1 Gbps link that would take about 4.6 days for 50 TB (theoretical max, but actual throughput will be lower due to overhead). Snowball Edge can transfer 50 TB in a few days and is cost-effective for large data volumes. Option A (AWS DataSync) is efficient for online transfers but may not meet the 5-day deadline over 1 Gbps. Option B (Amazon S3 Transfer Acceleration) speeds up transfers but still limited by network bandwidth. Option D (AWS Direct Connect) would require additional setup and cost, and still limited by the 1 Gbps link.

Answer

Use Amazon S3 Transfer Acceleration to speed up the transfer over the internet.

Answer

Use AWS DataSync to transfer the data over the existing network link.

Answer

Use AWS Direct Connect to establish a dedicated network connection.

Question 15

A company is ingesting streaming data from multiple sources using Amazon Kinesis Data Streams. The data is then processed by an AWS Lambda function that transforms the records and writes them to an Amazon S3 bucket. The Lambda function is failing intermittently with timeout errors. The average record size is 5 KB, and the shard count is 2. What is the MOST likely cause of the timeout errors?

Accepted Answer

The Lambda function timeout is set too low for the processing time required.. Option B is correct because the default Lambda timeout is 3 seconds, which may not be sufficient for processing and writing records, especially if the transformation logic is complex or the S3 write takes longer than expected. Option A is incorrect because Kinesis Data Streams has a default retention period of 24 hours, which is unlikely to cause timeouts. Option C is incorrect because the batch size of records per Lambda invocation can be adjusted, but the default is 100 records, which at 5 KB each is only 500 KB, well within Lambda limits. Option D is incorrect because Lambda concurrency limits affect scaling, not individual invocation timeouts.

Answer

The Kinesis data retention period is too short, causing data to be lost before processing.

Answer

The Lambda function's reserved concurrency is set too low, causing throttling.

Answer

The Lambda function is receiving too many records per invocation, exceeding the 6 MB payload limit.