DEA-C01 Data Ingestion and Transformation Practice Test 31 — 15 Questions

Question 1

A company uses AWS DMS to replicate data from an Amazon RDS for MySQL database to Amazon S3. Which TWO configurations are required to enable continuous change data capture (CDC) from MySQL?

Accepted Answer

Grant REPLICATION CLIENT and REPLICATION SLAVE privileges to the DMS user. Correct options: B and C. For AWS DMS to perform continuous change data capture (CDC) from a MySQL source, binary logging (binlog) must be enabled on the source database (option C) to capture changes. Additionally, the MySQL user used by DMS must be granted the REPLICATION CLIENT and REPLICATION SLAVE privileges (option B) to read the binlog and stream changes. Option D (S3 bucket versioning) is not required for DMS CDC. Option A (same Region) is not a requirement. Option E (Multi-AZ) is not needed for CDC. Therefore, B and C are the required configurations.

Answer

Ensure the S3 bucket is in the same AWS Region as the source database

Answer

Enable versioning on the target S3 bucket

Answer

Configure the MySQL source to be Multi-AZ

Question 2

A data engineer is designing a data pipeline that uses AWS Glue to transform data stored in Amazon S3. The transformation logic must be written in Python and should handle schema evolution automatically. Which THREE features or configurations should the engineer use? (Select THREE.)

Accepted Answer

Use `applyMapping` transformations. Correct options: B, D, E. AWS Glue DynamicFrames (E) handle schema evolution automatically by allowing schema on read and accommodating changes in data structure. Schema detection in the Glue job (D) enables the job to infer the schema from the data, which is essential for handling evolving schemas. Using `applyMapping` (B) provides explicit control over schema transformations and can be combined with DynamicFrames to manage schema changes. Option A (scheduling a Glue crawler) is meant for updating the Data Catalog, not for within-job schema evolution. Option C (Spark SQL) does not inherently handle schema evolution; it relies on static schemas.

Answer

Schedule a Glue crawler to update the schema

Answer

Use Spark SQL for transformations

Question 3

A company is using AWS Glue to process streaming data from Amazon Kinesis Data Streams. The job fails intermittently with a 'MemoryError' when the stream has a sudden spike in data volume. Which configuration change would best prevent this error?

Accepted Answer

Increase the number of DPUs (Data Processing Units) for the Glue job.. Increasing the number of DPUs in the AWS Glue job provides more memory and compute capacity to handle data spikes. Option B is wrong because storing intermediate results in Amazon RDS does not prevent memory errors in Glue; it introduces a database dependency and does not increase Glue's memory. Option C is wrong because switching to batch transformation is not a solution for a streaming job; the job is designed for streaming and batch does not address the memory issue. Option D is wrong because increasing the number of shards in Kinesis increases throughput but does not directly solve memory errors in Glue; it may even increase the data volume per unit time and worsen the problem.

Answer

Store intermediate results in Amazon RDS.

Answer

Use a batch transformation instead of streaming.

Answer

Increase the number of shards in the Kinesis data stream.

Question 4

A data engineer needs to ingest on-premises CSV files into Amazon S3 every hour. The files are less than 1 GB each. Which service is the most cost-effective and requires the least operational overhead?

Accepted Answer

AWS DataSync. AWS DataSync is the most cost-effective and least overhead option for scheduled, recurring transfers of on-premises CSV files to S3. It provides a simple agent-based setup, supports hourly scheduling, and handles files under 1GB efficiently without complex configuration. In contrast, Amazon Kinesis Data Firehose is designed for streaming data ingestion, not batch file transfers; AWS Snowball Edge is intended for large-scale offline data migrations, not hourly incremental transfers; and AWS Database Migration Service (DMS) is specialized for migrating databases, not file transfers.

Answer

Amazon Kinesis Data Firehose

Answer

AWS Snowball Edge

Answer

AWS Database Migration Service (DMS)

Question 5

A company uses AWS Glue to transform data in Amazon S3. The transformation logic is written in Python and references several libraries that are not included in the default Glue environment. Which approach should the data engineer use to make these libraries available?

Accepted Answer

Upload the libraries as a .zip file to an S3 bucket and reference them in the Glue job's Python library path.. AWS Glue allows you to provide custom Python libraries by uploading them as a .zip file to an S3 bucket and referencing the S3 path in the 'Python library path' field of the job. This method is supported for both Glue ETL and Python shell jobs, making custom libraries available at runtime. Option A is incorrect because Glue does not execute arbitrary shell commands like 'pip install' during job initialization; the job script runs in a controlled environment. Option B is incorrect because AWS Lambda layers are specific to Lambda functions and cannot be attached to Glue jobs. Option D is incorrect because the --additional-python-modules parameter, while valid in Glue 3.0 and later, only supports installing packages from PyPI, not custom libraries not published on PyPI.

Answer

Include a requirements.txt file in the Glue job script and run pip install during job initialization.

Answer

Package the libraries in an AWS Lambda layer and attach it to the Glue job.

Answer

Use the --additional-python-modules parameter in the Glue job.

Question 6

A data engineer needs to ingest data from an Amazon RDS for MySQL database into Amazon S3 on a daily basis. The data volume is about 50 GB per day. The engineer wants to minimize the impact on the source database. Which AWS service should be used?

Accepted Answer

AWS Database Migration Service (DMS). AWS DMS can perform full load and ongoing replication with minimal impact on the source database. Option A is wrong because AWS Glue with a JDBC connection can impact the source due to high query load. Option B is wrong because Amazon Athena Federated Query reads data directly from RDS, which can cause performance issues. Option D is wrong because AWS DataSync is designed for file storage, not databases.

Answer

AWS Glue with a JDBC connection

Answer

Amazon Athena Federated Query

Answer

AWS DataSync

Question 7

An e-commerce company wants to capture clickstream data from its website and store it in Amazon S3 for analytics. The data arrives continuously and the company needs near-real-time processing. Which solution is most appropriate?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the most appropriate solution because it is a fully managed service designed to ingest streaming data and deliver it to destinations like Amazon S3 with near-real-time latency. The company needs continuous clickstream capture and near-real-time processing, which Firehose provides. Option A (AWS Data Pipeline) is for batch processing, not streaming. Option B (AWS Snowball Edge) is for offline data transfer, not real-time. Option D (S3 Transfer Acceleration) improves upload speed but is not a streaming ingestion service.

Answer

AWS Data Pipeline

Answer

AWS Snowball Edge

Answer

Amazon S3 Transfer Acceleration

Question 8

A data engineer is designing a data ingestion pipeline for IoT sensor data. The sensors send JSON messages every second. The data must be available in Amazon S3 within 5 minutes and must be transformed (JSON to Parquet) before storage. Which combination of services meets these requirements?

Accepted Answer

Amazon Kinesis Data Firehose with data transformation and Parquet conversion. Amazon Kinesis Data Firehose can ingest streaming data, apply a transformation (e.g., convert JSON to Parquet), and deliver the transformed data to Amazon S3 with a buffer interval of up to 60 seconds, easily meeting the 5-minute latency requirement. Option A is incorrect because AWS Glue streaming ETL adds complexity and is not necessary for simple JSON-to-Parquet conversion; Kinesis Data Firehose handles this natively. Option C is incorrect because Kinesis Data Analytics is designed for real-time analytics and does not directly output to S3 in a transformed format without additional components. Option D is incorrect because S3 Event Notifications to Lambda would incur impractically high invocation costs and latency for per-second sensor data, and transforming on write to S3 would exceed the 5-minute window.

Answer

Amazon Kinesis Data Streams with AWS Glue streaming ETL

Answer

Amazon Kinesis Data Analytics with output to S3

Answer

Amazon S3 with S3 Event Notifications to AWS Lambda for transformation

Question 9

A company uses AWS Glue crawlers to populate the Data Catalog from data in Amazon S3. The crawler fails to update the schema when new columns are added to the CSV files. What is the most likely cause?

Accepted Answer

The crawler uses a custom classifier that defines a fixed schema.. When a crawler uses a custom classifier that defines a fixed schema, it will not recognize new columns added to the CSV files because the custom classifier overrides the schema inference. Option A is incorrect because S3 bucket versioning does not affect crawler schema updates. Option B is incorrect because the crawler configuration to crawl new partitions does not prevent schema updates; it only affects which partitions are crawled. Option C is incorrect because the IAM role would cause permissions errors, not a failure to update schema due to new columns.

Answer

The S3 bucket has versioning enabled.

Answer

The crawler is configured to only crawl new partitions.

Answer

The IAM role for the crawler lacks permissions to read the new columns.

Question 10

A company needs to transfer 20 TB of historical data from an on-premises Hadoop cluster to Amazon S3. The network bandwidth is limited and the transfer must complete within one week. Which service should the company use?

Accepted Answer

AWS Snowball Edge. AWS Snowball Edge is designed for large-scale data transfers (up to 80 TB per device) over limited bandwidth. It physically ships the data, bypassing network constraints entirely. The requirement of 20 TB within one week with limited bandwidth makes Snowball the ideal choice. Option A is wrong because S3 Transfer Acceleration still uses the internet and cannot guarantee completion within a week with limited bandwidth. Option C is wrong because Direct Connect provides a dedicated network connection but requires sufficient bandwidth; if bandwidth is limited, it cannot complete the transfer in time. Option D is wrong because DataSync over VPN still relies on the internet bandwidth and would not meet the deadline.

Answer

Amazon S3 Transfer Acceleration

Answer

AWS Direct Connect with DataSync

Answer

AWS DataSync over a VPN connection

Question 11

Which TWO AWS services can be used to ingest streaming data into Amazon S3 with minimal code? (Choose two.)

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose (Option B) is a fully managed service that can ingest streaming data and deliver it to Amazon S3 with minimal configuration and no code required. Amazon MSK with the S3 sink connector (Option C) allows streaming data from Apache Kafka topics to be automatically written to S3 with minimal code, as the connector handles the integration. Option A (AWS Lambda) requires custom code to process and write data to S3. Option D (AWS DMS) is designed for database migration, not streaming ingestion. Option E (AWS DataSync) is for batch file transfers, not real-time streaming.

Answer

AWS Lambda

Answer

AWS Database Migration Service (DMS)

Answer

AWS DataSync

Question 12

Which THREE factors should a data engineer consider when choosing between AWS Glue and Amazon EMR for a data transformation job? (Choose three.)

Accepted Answer

The level of control over the execution environment and dependencies. When choosing between AWS Glue and Amazon EMR for a data transformation job, key considerations include: the level of control over the execution environment (EMR offers more customization, while Glue is managed), the deployment model (Glue is serverless, EMR is cluster-based), and the cost structure (Glue charges per DPU, EMR charges per EC2 instance). Options A and B are not differentiating factors because both services support Apache Spark and can output to S3.

Answer

The ability to output results to Amazon S3

Answer

The support for Apache Spark

Question 13

Which TWO actions can improve the performance of an AWS Glue ETL job that processes large datasets in Amazon S3? (Choose two.)

Accepted Answer

Increase the number of DPUs allocated to the job.. Increasing the number of DPUs allocates more processing power to the Glue job, which can speed up data processing for large datasets. Option D is correct because columnar file formats like Parquet or ORC are more efficient for analytical queries, reduce I/O, and allow better compression compared to row-based formats. Option A is incorrect: increasing crawler frequency only affects the metadata catalog update frequency, not the ETL job performance. Option B is incorrect: using a single Availability Zone for the S3 bucket does not improve performance and may reduce availability. Option E is incorrect: using a single large file can reduce parallelism, as distributed processing benefits from splitting data into multiple files to be processed in parallel by different executors.

Answer

Increase the frequency of the Glue crawler.

Answer

Use a single Availability Zone for the S3 bucket.

Answer

Use a single large file instead of many small files.

Question 14

Refer to the exhibit. An IAM policy is attached to a role used by an AWS Glue job. The job fails with an 'AccessDenied' error when trying to write to 's3://my-bucket/output/'. What is the most likely cause?

Accepted Answer

The resource ARN for S3 should include the bucket itself.. The job fails with 'AccessDenied' when writing to S3. This is most likely because the IAM policy attached to the Glue role does not grant the s3:PutObject permission on the object ARN (e.g., 'arn:aws:s3:::my-bucket/output/*'). Option A points out that the resource ARN should include the bucket itself, which is a common error: if the policy uses the bucket ARN (without the /* suffix) for the PutObject action, it will not allow the write operation. Option C is incorrect because s3:ListBucket is not required for the PutObject action; it is needed for listing contents, not for writing. Option B is irrelevant to the S3 write failure. Option D (s3:GetObject) is for reading, not writing.

Answer

The glue:StartJobRun action is not allowed.

Answer

The policy does not grant s3:ListBucket permission.

Answer

The s3:GetObject action is missing.

Question 15

Refer to the exhibit. A data engineer runs this AWS CLI command to create a Glue job. The job processes JSON files in an S3 bucket and writes Parquet files to another bucket. After the first successful run, the job re-processes all input files instead of only new files. What is the most likely cause?

Accepted Answer

The job script does not implement job bookmark support.. The command sets '--job-bookmark-enable' but if the job script does not use the bookmark APIs or implement bookmark support, Glue will not track processed files and will reprocess all input on each run. Option A is incorrect because the ScriptLocation is valid and does not affect bookmark behavior. Option B is incorrect because max-retries does not control reprocessing. Option D is incorrect because the IAM role is specified and permissions for bookmark state are not explicitly shown, but the lack of bookmark support in the script is the issue.

Answer

The ScriptLocation points to an incorrect S3 path.

Answer

The --max-retries parameter is set to 0.

Answer

The IAM role lacks permissions to read bookmark state.