DEA-C01 Data Ingestion and Transformation Practice Test 30 — 15 Questions

Question 1

A company needs to ingest data from an on-premises SQL Server database into Amazon Redshift. The data volume is less than 1 TB and the network bandwidth is limited. Which AWS service should be used for the initial full load?

Accepted Answer

AWS Database Migration Service (DMS). Option B is correct because AWS DMS is designed for migrating databases to AWS, including to Redshift. Option A (AWS Snowball) is for large data volumes (petabytes) and not efficient for <1 TB. Option C (Amazon S3 Transfer Acceleration) speeds up uploads to S3 but not directly to Redshift. Option D (AWS Direct Connect) is a network connection, not a migration service.

Answer

AWS Snowball Edge

Answer

Amazon S3 Transfer Acceleration

Answer

AWS Direct Connect

Question 2

A data engineer is designing a data ingestion pipeline for real-time clickstream data using Amazon Kinesis Data Streams. The data must be transformed using AWS Lambda and then stored in Amazon S3 in Parquet format. Which Kinesis client library configuration should be used to minimize the number of Lambda invocations while ensuring data is processed within 60 seconds?

Accepted Answer

Set batch size to 10000 records and batch window to 60 seconds. Option B is correct because setting a larger batch window (up to 300 seconds) reduces invocations; 60 seconds is acceptable. Option A (smaller batch size) increases invocations. Option C (smaller batch window) increases invocations. Option D (disable retries) is not recommended.

Answer

Set batch size to 100 records and disable batch window

Answer

Set batch size to 100 records and batch window to 0 seconds

Answer

Set batch size to 10000 records and batch window to 5 seconds

Question 3

A company is using Amazon MSK (Managed Streaming for Apache Kafka) to ingest real-time data. They need to transform the data using custom Java code before writing to Amazon S3. The transformation must be fault-tolerant and exactly-once semantics are required. Which AWS service should be used?

Accepted Answer

Kafka Connect with S3 Sink Connector. Option C is correct because Kafka Connect with S3 Sink Connector supports exactly-once semantics and custom transformations via Single Message Transforms (SMTs) or custom connectors. Option A (Kinesis Data Analytics for Apache Flink) is for Flink, not Kafka. Option B (AWS Lambda) does not provide exactly-once semantics from MSK. Option D (Amazon EMR) is for batch processing.

Answer

Amazon EMR with Spark Streaming

Answer

AWS Lambda consumer for MSK

Answer

Kinesis Data Analytics for Apache Flink

Question 4

A company needs to ingest data from Amazon S3 into Amazon Redshift for analytics. The data arrives in CSV format with headers and may contain duplicate rows. Which Redshift command should be used to load the data while handling duplicates?

Accepted Answer

Use a staging table with COPY and then MERGE into the target table. Option B is correct because the `COPY` command with the `REMOVEDUPLICATES` option is not valid; the correct approach is to use a staging table and then use `MERGE` (or `UPSERT`) to handle duplicates. Option A (COPY) alone does not remove duplicates. Option C (INSERT INTO SELECT) does not handle duplicates. Option D (CREATE TABLE AS) does not handle duplicates.

Answer

COPY command with the `REMOVEDUPLICATES` option

Answer

INSERT INTO ... SELECT DISTINCT from S3 via Spectrum

Answer

CREATE TABLE AS SELECT DISTINCT from the S3 bucket

Question 5

A company is building a data lake on Amazon S3 and needs to ingest data from multiple sources. Which of the following AWS services can be used to ingest and transform data in near real-time? (Select TWO.)

Accepted Answer

AWS Glue. Correct options: A and C. Kinesis Data Firehose can ingest streaming data and perform basic transformations. AWS Glue can be triggered by S3 events for near real-time transformation. Option B (Amazon Athena) is a query service, not ingestion. Option D (Amazon SQS) is a message queue. Option E (AWS Step Functions) is for orchestration.

Answer

Amazon Athena

Answer

AWS Step Functions

Answer

Amazon Simple Queue Service (SQS)

Question 6

A company uses AWS DMS to replicate data from an Amazon RDS for MySQL database to Amazon S3. Which TWO configurations are required to enable continuous change data capture (CDC) from MySQL?

Accepted Answer

Grant REPLICATION CLIENT and REPLICATION SLAVE privileges to the DMS user. Correct options: B and D. DMS CDC requires binary logs (binlog) to be enabled on the MySQL source and the S3 target endpoint must be configured with `DataFormat` set to `parquet` or `csv` (but not required for CDC). However, to capture changes, binlog retention must be sufficient. Option B is correct: binlog must be enabled. Option D is correct: DMS needs the `REPLICATION CLIENT` and `REPLICATION SLAVE` privileges. Option A is incorrect; the S3 bucket versioning is not required. Option C is incorrect; the source database does not need to be in Multi-AZ for CDC. Option E is incorrect; the target S3 bucket does not need to be in the same region (though recommended).

Answer

Ensure the S3 bucket is in the same AWS Region as the source database

Answer

Enable versioning on the target S3 bucket

Answer

Configure the MySQL source to be Multi-AZ

Question 7

A data engineer is designing a data pipeline that uses AWS Glue to transform data stored in Amazon S3. The transformation logic must be written in Python and should handle schema evolution automatically. Which THREE features or configurations should the engineer use? (Select THREE.)

Accepted Answer

Use `applyMapping` transformations. Correct options: A, B, C. AWS Glue DynamicFrames handle schema evolution automatically. Glue's schema detection can infer schema from data. Using `applyMapping` allows for schema transformations. Option D (Spark SQL) does not handle schema evolution automatically. Option E (Glue crawlers) are for cataloging, not transformation.

Answer

Schedule a Glue crawler to update the schema

Answer

Use Spark SQL for transformations

Question 8

A company is using AWS Glue to process streaming data from Amazon Kinesis Data Streams. The job fails intermittently with a 'MemoryError' when the stream has a sudden spike in data volume. Which configuration change would best prevent this error?

Accepted Answer

Increase the number of DPUs (Data Processing Units) for the Glue job.. Option A is correct because increasing the number of DPUs in the AWS Glue job provides more memory for processing spikes. Option B is wrong because Kinesis shard count affects throughput, not memory. Option C is wrong because batch processing does not help streaming jobs. Option D is wrong because RDS is unrelated to Glue memory.

Answer

Store intermediate results in Amazon RDS.

Answer

Use a batch transformation instead of streaming.

Answer

Increase the number of shards in the Kinesis data stream.

Question 9

A data engineer needs to ingest on-premises CSV files into Amazon S3 every hour. The files are less than 1 GB each. Which service is the most cost-effective and requires the least operational overhead?

Accepted Answer

AWS DataSync. Option B is correct because AWS DataSync is designed for scheduled transfers from on-premises to S3 with minimal setup. Option A is wrong because Snowball is for large, offline transfers. Option C is wrong because Kinesis is for streaming data. Option D is wrong because DMS is for database migrations.

Answer

Amazon Kinesis Data Firehose

Answer

AWS Snowball Edge

Answer

AWS Database Migration Service (DMS)

Question 10

A company uses AWS Glue to transform data in Amazon S3. The transformation logic is written in Python and references several libraries that are not included in the default Glue environment. Which approach should the data engineer use to make these libraries available?

Accepted Answer

Upload the libraries as a .zip file to an S3 bucket and reference them in the Glue job's Python library path.. Option C is correct because AWS Glue supports Python shell jobs with extra Python modules via S3 paths. Option A is wrong because --additional-python-modules is not valid. Option B is wrong because Lambda has a deployment package limit. Option D is wrong because Glue does not support pip install directly.

Answer

Include a requirements.txt file in the Glue job script and run pip install during job initialization.

Answer

Package the libraries in an AWS Lambda layer and attach it to the Glue job.

Answer

Use the --additional-python-modules parameter in the Glue job.

Question 11

A data engineer needs to ingest data from an Amazon RDS for MySQL database into Amazon S3 on a daily basis. The data volume is about 50 GB per day. The engineer wants to minimize the impact on the source database. Which AWS service should be used?

Accepted Answer

AWS Database Migration Service (DMS). Option D is correct because AWS DMS can perform full load and ongoing replication with minimal impact. Option A is wrong because Glue's JDBC connection can impact the source. Option B is wrong because DataSync is for file storage. Option C is wrong because Athena cannot read from RDS directly.

Answer

AWS Glue with a JDBC connection

Answer

Amazon Athena Federated Query

Answer

AWS DataSync

Question 12

An e-commerce company wants to capture clickstream data from its website and store it in Amazon S3 for analytics. The data arrives continuously and the company needs near-real-time processing. Which solution is most appropriate?

Accepted Answer

Amazon Kinesis Data Firehose. Option A is correct because Kinesis Data Firehose is a fully managed service for loading streaming data into S3 with near-real-time delivery. Option B is wrong because S3 Transfer Acceleration is for faster uploads, not streaming. Option C is wrong because Data Pipeline is for batch processing. Option D is wrong because Snowball is offline.

Answer

AWS Data Pipeline

Answer

AWS Snowball Edge

Answer

Amazon S3 Transfer Acceleration

Question 13

A data engineer is designing a data ingestion pipeline for IoT sensor data. The sensors send JSON messages every second. The data must be available in Amazon S3 within 5 minutes and must be transformed (JSON to Parquet) before storage. Which combination of services meets these requirements?

Accepted Answer

Amazon Kinesis Data Firehose with data transformation and Parquet conversion. Option C is correct because Kinesis Firehose can ingest streaming data and transform it to Parquet before delivering to S3. Option A is wrong because Lambda from S3 would cause delay. Option B is wrong because Glue streaming jobs add complexity. Option D is wrong because Kinesis Data Analytics does not output to S3 directly.

Answer

Amazon Kinesis Data Streams with AWS Glue streaming ETL

Answer

Amazon Kinesis Data Analytics with output to S3

Answer

Amazon S3 with S3 Event Notifications to AWS Lambda for transformation

Question 14

A company uses AWS Glue crawlers to populate the Data Catalog from data in Amazon S3. The crawler fails to update the schema when new columns are added to the CSV files. What is the most likely cause?

Accepted Answer

The crawler uses a custom classifier that defines a fixed schema.. Option A is correct because if the crawler is configured with a custom classifier that expects a fixed schema, it will ignore new columns. Option B is wrong because partition discovery does not prevent schema updates. Option C is wrong because S3 event notifications are not related. Option D is wrong because IAM permissions would cause a different error.

Answer

The S3 bucket has versioning enabled.

Answer

The crawler is configured to only crawl new partitions.

Answer

The IAM role for the crawler lacks permissions to read the new columns.

Question 15

A data engineer is ingesting data from a third-party API into Amazon S3 using AWS Lambda. The API returns a JSON payload of up to 10 MB per request. The Lambda function runs every minute. Occasionally, the function times out after 15 seconds. What is the most likely cause?

Accepted Answer

The Lambda function's timeout setting is too short.. Option B is correct because Lambda has a default timeout of 3 seconds, but can be increased to 15 minutes. If the function times out at 15 seconds, the timeout is set to 15 seconds. Increasing it resolves the issue. Option A is wrong because 10 MB is within Lambda limits. Option C is wrong because memory may need increase but timeout is the immediate cause. Option D is wrong because VPC configuration can cause delays but not specifically timeout.

Answer

The Lambda function is in a VPC without a NAT gateway, causing network timeouts.

Answer

The Lambda function's memory is too low, causing slow processing.

Answer

The API response size exceeds the Lambda invocation payload limit.