DEA-C01 Data Ingestion and Transformation Practice Test 39 — 15 Questions

Question 1

Refer to the exhibit. A data engineer is using a Kinesis Data Stream with 2 shards. The producer uses a partition key that is the user ID (a UUID). The consumer is falling behind. Which change would improve throughput?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Increase the number of shards. The consumer is falling behind because the total throughput of the stream (1 MB/s or 1,000 records/s per shard for writes, and 2 MB/s per shard for reads) is insufficient for the incoming data volume. Increasing the number of shards scales both the write and read capacity linearly, allowing the consumer to process records faster and catch up. Changing the partition key or retention period does not increase throughput, and switching to Firehose changes the delivery model but does not inherently solve the consumer lag.

Answer

Switch to Kinesis Data Firehose

Answer

Increase the retention period

Answer

Change the partition key to a constant value

Question 2

A data engineer is designing a data ingestion pipeline to load millions of small JSON files from an on-premises FTP server into Amazon S3. The pipeline should minimize cost and operational overhead. Which approach is most suitable?

Accepted Answer

Deploy AWS DataSync to transfer files from the FTP server to S3. AWS DataSync is the most suitable option because it is designed to efficiently transfer large volumes of data from on-premises storage (including FTP servers) to AWS, handling millions of small files with minimal operational overhead. It automates data transfer, retries, and validation, and it is cost-effective as you pay only for the data transferred, with no need for additional infrastructure or complex scripting.

Answer

Use S3 Transfer Acceleration to upload files directly from the FTP server

Answer

Use AWS Snowball Edge to ship the data to AWS

Answer

Set up an AWS Direct Connect connection and use AWS CLI to copy files

Question 3

A company uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The data is transformed using an AWS Lambda function. Recently, the transformation errors have increased due to Lambda timeouts. The data engineer needs to diagnose and resolve the issue without losing data. What should the engineer do?

Accepted Answer

Increase the Lambda function timeout and ensure that failed records are sent to a backup S3 bucket. Increasing the Lambda function timeout directly addresses the root cause of transformation errors (timeouts), and configuring a backup S3 bucket for failed records ensures no data loss. Kinesis Data Firehose can be configured to send failed records to a separate S3 bucket as a dead-letter queue, which preserves the data for later reprocessing while the primary transformation pipeline is fixed.

Answer

Enable Amazon CloudWatch Logs for the Lambda function to capture errors and store failed records in CloudWatch

Answer

Configure the Lambda function to write failed records to an Amazon SQS queue for later reprocessing

Answer

Modify the Lambda function to store failed records in Amazon S3 before processing

Question 4

A data engineer is building a streaming pipeline using Amazon Kinesis Data Streams and AWS Lambda. The Lambda function processes records and writes to Amazon DynamoDB. The engineer notices that the Lambda function is throttled during high traffic. Which action should the engineer take to reduce throttling?

Accepted Answer

Increase the number of shards in the Kinesis data stream. Increasing the number of shards in the Kinesis data stream increases the overall throughput of the stream, which allows the Lambda event source mapping to poll more shards concurrently. Each shard is processed by one Lambda invocation at a time, so more shards mean more concurrent Lambda executions, reducing the per-invocation load and the likelihood of throttling.

Answer

Increase the Lambda function timeout

Answer

Disable retries on the Lambda function

Answer

Use an Amazon SQS queue as an intermediate buffer

Question 5

A company needs to ingest data from multiple SaaS applications into Amazon S3. The data sources provide REST APIs. Which AWS service can be used to build a fully managed data ingestion pipeline without writing custom code?

Accepted Answer

Amazon AppFlow. Amazon AppFlow is a fully managed service designed to transfer data from SaaS applications to AWS services like Amazon S3 without writing custom code. Option B is wrong because Amazon Kinesis Data Streams is primarily for real-time streaming data, not for directly ingesting from SaaS APIs. Option C is wrong because AWS Lambda requires custom code to integrate with SaaS APIs. Option D is wrong because AWS Glue with Python shell is for ETL transformations and also requires custom scripting.

Answer

Amazon Kinesis Data Streams

Answer

AWS Lambda with custom code

Answer

AWS Glue with Python shell

Question 6

A company uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The data is transformed using an AWS Lambda function. Some records fail transformation and are lost because the Lambda function throws an exception. The data engineer needs to capture the failed records for analysis without affecting the pipeline. What should the engineer do?

Accepted Answer

Configure the Firehose delivery stream to send failed records to a backup S3 bucket. Amazon Kinesis Data Firehose natively supports a 'backup' or 'error output' configuration: when a Lambda transformation fails (e.g., throws an exception), Firehose can automatically route the failed records to a designated Amazon S3 bucket for failed data. This allows the engineer to capture and analyze the failed records without blocking or altering the main delivery pipeline, as the stream continues processing successfully transformed records.

Answer

Increase the buffer size of the Firehose stream

Answer

Disable the Lambda transformation and process all records in batch later

Answer

Modify the Lambda function to write failed records to Amazon DynamoDB

Question 7

A data engineer is building a pipeline to ingest data from an on-premises Oracle database into Amazon S3. The pipeline must capture change data (CDC) in near real-time and handle schema changes. Which TWO AWS services should the engineer use?

Accepted Answer

AWS Glue Schema Registry. AWS Glue Schema Registry (A) is correct because it enables schema discovery, validation, and evolution for streaming data, allowing the pipeline to handle schema changes from the Oracle CDC source. It integrates with Apache Kafka and Amazon Kinesis Data Streams to enforce schema compatibility rules (e.g., backward, forward, full) as data arrives, ensuring downstream consumers can adapt to evolving schemas without breaking.

Answer

AWS Snowball Edge

Answer

Amazon AppFlow

Answer

Amazon Kinesis Data Streams with Kinesis Agent

Question 8

Refer to the exhibit. A data engineer is using a Kinesis Data Stream with 2 shards. The producer uses a partition key that is the user ID (a UUID). The consumer is falling behind. Which change would improve throughput?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The stream has only one shard, which limits writes to 1000 records per second. A Kinesis Data Stream shard has a write throughput limit of 1,000 records per second (or 1 MB per second). Since the application is writing 2,000 records per second (each 1 KB) to a single shard, it exceeds the shard's record-per-second quota, causing the PutRecord calls to be throttled. The solution is to increase the number of shards to at least two to distribute the load.

Answer

The retention period of 24 hours is too short

Answer

The stream uses KMS encryption, causing additional latency

Answer

Enhanced monitoring is not enabled, causing performance issues

Question 9

Refer to the exhibit. An S3 event notification is configured to trigger an AWS Lambda function when objects are created in 'my-bucket'. The Lambda function processes the JSON file and writes results to Amazon DynamoDB. The function fails with a timeout error. Which action should the engineer take to resolve the issue?

Accepted Answer

Increase the Lambda function timeout. The Lambda function is failing with a timeout error, which indicates that the function is taking longer to execute than the default timeout of 3 seconds. Increasing the Lambda function timeout allows the function to run longer and complete its processing of the JSON file and DynamoDB write operation without being prematurely terminated.

Answer

Modify the S3 event notification to use a different event type

Answer

Grant the Lambda function permission to access DynamoDB

Answer

Change the trigger to Amazon SQS instead of S3

Question 10

A company uses Amazon S3 to store raw data and needs to transform it into Parquet format for analytics. The transformation job runs daily on a schedule. Which AWS service is BEST suited for this task?

Accepted Answer

AWS Glue. AWS Glue is a fully managed, serverless ETL service that can automatically convert data formats (e.g., from CSV to Parquet) and run on a schedule (e.g., daily). It is ideal for this use case because it is purpose-built for ETL transformations and handles schema discovery, data cataloging, and job scheduling without managing infrastructure. Option A (Amazon Redshift) is wrong because Redshift is a data warehouse for querying, not a transformation service; it could load Parquet but not convert raw data to Parquet directly. Option B (Amazon EMR) is wrong because EMR requires provisioning and managing clusters, adding operational overhead. Option C (AWS Lambda) is wrong because Lambda has a maximum execution timeout of 15 minutes, which is too short for daily large-scale data transformation jobs.

Answer

Amazon Redshift

Answer

Amazon EMR

Answer

AWS Lambda

Question 11

A company wants to import data from an external FTP server into Amazon S3 on a daily basis. The data volumes are moderate. Which AWS service is MOST suitable for this task?

Accepted Answer

AWS Transfer Family. AWS Transfer Family is the most suitable service because it provides fully managed support for SFTP, FTPS, and FTP protocols, enabling direct, secure file transfers from an external FTP server to Amazon S3 without needing to manage any infrastructure. It integrates natively with S3, so files are automatically stored in the specified bucket upon transfer completion, making it ideal for daily imports of moderate data volumes.

Answer

Amazon S3 Transfer Acceleration

Answer

AWS DataSync

Answer

AWS Glue with a JDBC connection

Question 12

A company has a 100 TB dataset stored on-premises in a Hadoop cluster. They want to ingest this data into Amazon S3 for processing with AWS Glue. The company has a limited time window and a slow internet connection. Which strategy is MOST appropriate?

Accepted Answer

Use AWS Snowball Edge to physically ship the data to AWS.. AWS Snowball Edge is the most appropriate strategy because the dataset is 100 TB, the time window is limited, and the internet connection is slow. Snowball Edge provides a physical storage device that can be shipped to AWS, bypassing network bandwidth constraints entirely. This approach is designed for large-scale data transfers (typically over 10 TB) where network transfer would be impractical or exceed the available time window.

Answer

Use AWS DataSync over the existing internet connection.

Answer

Use Amazon S3 Transfer Acceleration to speed up the upload.

Answer

Use AWS Direct Connect to establish a high-bandwidth connection.

Question 13

A data engineer needs to ingest streaming data from a social media API into Amazon S3 for batch analytics. The data arrives at a rate of 500 records per second. Which service should be used to capture the stream?

Accepted Answer

Amazon Kinesis Data Streams. Amazon Kinesis Data Streams is designed for real-time streaming data ingestion at scale, supporting throughput of up to 1 MB/s or 1,000 records per second per shard. With 500 records per second, Kinesis can reliably capture and store the social media API data for up to 365 days, enabling batch analytics via S3 delivery through Kinesis Firehose or custom consumers.

Answer

Amazon Simple Notification Service (SNS)

Answer

Amazon Simple Queue Service (SQS)

Answer

Amazon MQ

Question 14

A company is designing a data ingestion pipeline for real-time sensor data from thousands of devices. The data must be processed with low latency and stored in Amazon S3. Which TWO services would be appropriate for this use case? (Choose TWO.)

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service designed to ingest real-time streaming data, transform it on the fly (e.g., convert to Parquet/ORC), and deliver it directly to Amazon S3 with low latency. It handles buffering, compression, and partitioning automatically, making it ideal for the described sensor data pipeline.

Answer

AWS Glue

Answer

AWS DataSync

Answer

Amazon Athena

Question 15

A data engineer runs the describe-stream command and sees the output above. The stream has a retention period of 24 hours. The engineer needs to ensure that consumers can replay data for up to 7 days. Which action is required?

Accepted Answer

Use the IncreaseStreamRetentionPeriod API to set retention to 168 hours.. The describe-stream output shows a retention period of 24 hours, but the requirement is to allow consumers to replay data for up to 7 days (168 hours). Amazon Kinesis Data Streams supports modifying the retention period dynamically without recreating the stream, using the IncreaseStreamRetentionPeriod API or the update-shard-count command. Option C correctly uses this API to set retention to 168 hours, which is the maximum supported retention period for Kinesis Data Streams.

Answer

Increase the number of shards to allow more data storage.

Answer

Delete the stream and recreate it with a longer retention period.

Answer

Create new consumer applications that read from the stream.