DEA-C01 Data Ingestion and Transformation Practice Test 7 — 15 Questions

Question 1

A data engineer needs to ingest data from an on-premises Oracle database into Amazon S3 on a daily basis. The data volume is approximately 500 GB per day. The source database is behind a firewall that does not allow direct internet access. Which service should the engineer use to transfer the data securely?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

AWS DataSync with a network path through AWS Direct Connect or VPN.. AWS DataSync can transfer data from on-premises storage to AWS over a network path established through AWS Direct Connect or a VPN, which is necessary because the source database is behind a firewall that blocks direct internet access. DataSync automates the movement of large datasets (up to 500 GB daily) and integrates with Oracle databases via a supported agent, handling incremental transfers efficiently. This makes it the correct choice for secure, scheduled, and high-volume data ingestion into Amazon S3.

Answer

AWS Database Migration Service (AWS DMS) with ongoing replication from Oracle to S3.

Answer

Amazon S3 Transfer Acceleration with a public endpoint.

Answer

AWS Snowball Edge device for daily transfers.

Question 2

A company uses AWS Glue to transform data stored in Amazon S3. During a run, the job fails with a 'OutOfMemoryError' in the Spark executor. The job processes 2 TB of parquet files using 10 DPUs. The data is evenly distributed across partitions. Which action would MOST likely resolve the issue without impacting the job logic?

Accepted Answer

Increase the number of DPUs allocated to the Glue job.. The OutOfMemoryError in the Spark executor indicates that the available memory per executor is insufficient for the data being processed. Increasing the number of DPUs allocated to the Glue job increases the total memory and compute resources available, allowing Spark to handle the 2 TB dataset without changing the job logic.

Answer

Enable S3 request rate increase to speed up data reading.

Answer

Repartition the data to a larger number of partitions.

Answer

Change the input format from Parquet to Snappy-compressed CSV.

Question 3

A data engineer is building a batch ETL pipeline using AWS Glue. The source data is in Amazon RDS for MySQL. The pipeline must run daily and process only new and modified records since the last run. The engineer needs to implement change data capture (CDC) efficiently. Which THREE steps should the engineer take? (Choose THREE.)

Accepted Answer

Set a job bookmark in Glue to track processed records.. Option C is correct because AWS Glue job bookmarks persist state between runs and allow the job to process only new or changed data since the previous execution, which is essential for incremental CDC-style ETL. Option D is correct because MySQL change data capture relies on the binary log (binlog), so binary logging must be enabled on the RDS for MySQL instance to capture row-level changes. Option E is correct because a full table scan with a WHERE clause on a timestamp column (for example, filtering on updated_at or a similar column) is a valid and commonly used way to extract only records that are new or modified since the last run. Option A is not correct because AWS Glue JDBC connections cannot directly read the MySQL binary log via a SQL query; binlog access requires a CDC tool or service that interprets the log stream. Option B is not correct because streaming RDS data through Kinesis Data Streams is unnecessary for a daily batch Glue ETL job and does not by itself implement CDC for the batch scenario.

Answer

Configure the Glue job to use a JDBC connection with a SQL query that reads from the binary log.

Answer

Ingest the RDS data into Kinesis Data Streams and then use Glue.

Question 4

Refer to the exhibit. A data engineer is attaching this IAM policy to an IAM role used by an AWS Glue job. The job reads from a Kinesis Data Streams stream and writes transformed data to an S3 bucket. When the job runs, it fails with an AccessDenied error for the Kinesis stream. What is the MOST likely cause?

Accepted Answer

The IAM policy is missing the 'kinesis:DescribeStream' action.. When an AWS Glue job reads from Kinesis Data Streams, it requires the `kinesis:DescribeStream` action to retrieve stream metadata such as shard IDs and the stream's current state. Without this permission, the Glue job cannot discover the stream's shards and fails with an AccessDenied error, even if other Kinesis actions like `GetRecords` are allowed. The error occurs because the IAM policy attached to the Glue role is missing this prerequisite action.

Answer

The stream ARN in the policy is incorrect.

Answer

The Glue job does not have permissions to call 'kinesis:PutRecord'.

Answer

The S3 bucket policy blocks the PutObject action from the Glue role.

Question 5

Refer to the exhibit. A data engineer runs the describe-stream command on a Kinesis data stream. The stream has two shards. The engineer wants to increase the shard count to 4 using the UpdateShardCount API. What will be the resulting shard distribution?

Accepted Answer

Four shards with hash ranges: 0-5.67e37, 5.67e37-1.13e38, 1.13e38-1.70e38, 1.70e38-2.27e38.. When the UpdateShardCount API doubles the shard count from 2 to 4, Kinesis splits the existing hash range (0 to 2^128-1, approximately 3.40e38) into four equal contiguous sub-ranges. The precise boundaries are: 0, 5.67e37, 1.13e38, 1.70e38, and 2.27e38. Option B lists these exact boundaries, making it correct. Option D lists the same boundaries, but since it is a duplicate of the correct answer, it is considered a distractor and is not a valid separate correct choice. Options A and C contain incorrect values or descriptions.

Answer

Four shards with hash ranges: 0-5.68e37, 5.68e37-1.13e38, 1.13e38-1.70e38, 1.70e38-2.27e38.

Answer

Two new shards are created with overlapping hash ranges, and the original two shards remain as parents.

Answer

Four shards with hash ranges: 0-5.67e37, 5.67e37-1.13e38, 1.13e38-1.70e38, 1.70e38-2.27e38.

Question 6

Refer to the exhibit. A company uses S3 Event Notifications to trigger an AWS Lambda function whenever a new object is uploaded to an S3 bucket. The Lambda function processes the file and moves it to a different bucket. Recently, the function has been failing intermittently. The engineer checks the Lambda CloudWatch logs and sees the above event. What is the MOST likely cause of the intermittent failures?

Accepted Answer

The S3 event notifications are not guaranteed to be delivered exactly once, causing duplicate processing.. S3 Event Notifications are asynchronous and may be duplicated or delivered out of order, causing race conditions. Option A is wrong because the event has all required fields. Option B is wrong because the bucket name is valid. Option D is wrong because the event format is correct.

Answer

The event JSON is malformed; the 'EventSource' should be 's3.amazonaws.com'.

Answer

The S3 bucket name contains a hyphen, which is not allowed.

Answer

The event is missing the 'object:versionId' field.

Question 7

A data engineer uses AWS DMS to migrate a 2 TB PostgreSQL database to Amazon Aurora PostgreSQL. The migration task is set to full load + CDC. After the full load completes, the CDC phase starts but shows a high latency of 5 minutes. The source database has a low write load. What should the engineer do to reduce the CDC latency?

Accepted Answer

Increase the size of the DMS replication instance.. To reduce CDC latency, increasing the size of the DMS replication instance (Option C) is the most effective action because it provides more CPU and memory to process the change data capture stream. Option A (decreasing batch size) could actually worsen latency by increasing the number of batches. Option B (disabling validation) might reduce some overhead but does not address potential resource constraints. Option D (enabling logging) adds overhead and does not reduce latency.

Answer

Decrease the batch size for the CDC task.

Answer

Disable the validation feature on the DMS task.

Answer

Enable logging for the DMS task to capture additional details.

Question 8

A data engineer is designing a data ingestion pipeline for real-time user activity logs. The logs are generated by a web application and must be ingested into Amazon S3 with minimal latency (under 1 minute). The logs also need to be queried in Amazon Athena. The engineer considers using Amazon Kinesis Data Firehose. Which TWO configurations are required to achieve near-real-time delivery to S3? (Choose TWO.)

Accepted Answer

Set the BufferIntervalInSeconds to 60 seconds.. Setting BufferIntervalInSeconds to 60 seconds forces Kinesis Data Firehose to deliver data to S3 every 60 seconds, meeting the sub-1-minute latency requirement. Option E is correct because setting BufferSizeInMBs to 1 MB ensures that the buffer fills quickly and triggers delivery when the buffer reaches 1 MB, which, combined with the time-based trigger, minimizes latency.

Answer

Enable S3 compression (e.g., GZIP) on the delivery stream.

Answer

Enable Amazon CloudWatch error logging for the delivery stream.

Answer

Enable data format conversion to Parquet using AWS Glue.

Question 9

An organization needs to ingest data from on-premises databases into AWS S3 for archival purposes. The data volume is several TB per day, and the network has moderate bandwidth. Which AWS service is BEST suited for this bulk data transfer?

Accepted Answer

AWS DataSync. AWS DataSync is the best choice because it is designed for efficiently moving large volumes of data (several TB per day) from on-premises storage to AWS, including S3. It automates and accelerates data transfer over the existing network using a purpose-built protocol and parallel multi-threading, while handling encryption, validation, and scheduling. For moderate bandwidth, DataSync optimizes throughput and can resume interrupted transfers, making it ideal for ongoing archival ingestion.

Answer

AWS Direct Connect

Answer

Amazon S3 Transfer Acceleration

Answer

AWS Snowball

Question 10

A company uses AWS Glue for ETL jobs. The data engineer needs to ensure that the Glue job can access an S3 bucket in another account. What is the recommended approach?

Accepted Answer

Assign an IAM role to the Glue job with permissions to access the bucket, and configure the bucket policy to allow the role. The recommended approach is to give the Glue job an IAM role with permissions to access the S3 bucket and add a bucket policy in the target account that grants access to that role. This uses cross-account IAM role trust and resource-based policy, which is the AWS-recommended pattern for cross-account S3 access without long-lived credentials.

Answer

Create an IAM role in the target account and have the Glue job assume that role

Answer

Configure the S3 bucket policy to allow the Glue job's IAM role and also set the Glue job's resource-based policy

Answer

Store AWS access keys for the target account in AWS Secrets Manager and have the Glue job retrieve them

Question 11

A company uses Kinesis Data Streams to ingest clickstream data. They notice that the data processing latency increases as the number of shards grows. What is the most likely cause and solution?

Accepted Answer

Reduce the number of shards or increase the number of consumers.. As the number of shards increases, the total throughput of the stream increases, but each shard has a fixed limit of 5 read transactions per second. If the number of consumers remains constant, each consumer must poll more shards sequentially, increasing the per-record processing latency. Reducing the number of shards or adding more consumers distributes the polling load, reducing the time each consumer spends waiting for shard-level throttling.

Answer

Increase the Kinesis Producer Library (KPL) batch size.

Answer

Use enhanced fan-out to allow multiple consumers per shard.

Answer

Increase the number of shards to handle more data.

Question 12

A data engineer needs to transform CSV files to Parquet format using AWS Glue. The source data contains sensitive columns that must be masked. Which Glue feature should be used?

Accepted Answer

AWS Glue DataBrew. AWS Glue DataBrew is a visual data preparation tool that includes built-in transformations for masking sensitive data, such as hashing, tokenizing, or redacting columns. This makes it the correct choice for transforming CSV files to Parquet while applying column-level masking without writing custom code.

Answer

AWS Glue Studio

Answer

AWS Glue Crawler

Answer

AWS Glue Schema Registry

Question 13

A company uses AWS Database Migration Service (DMS) to continuously replicate data from an Oracle RDS instance to S3. The data is used for analytics. The replication lags behind the source by several hours. Which change would most likely reduce the lag?

Accepted Answer

Use a larger DMS replication instance (e.g., dms.c5.large instead of dms.t3.medium).. The replication lag is most likely caused by the DMS replication instance being undersized for the volume of change data capture (CDC) events. Upgrading from a burstable t3.medium to a compute-optimized c5.large instance provides more consistent CPU performance and higher network throughput, enabling faster processing of Oracle redo logs and reducing the lag between source and target.

Answer

Change the target endpoint from S3 to Kinesis Data Firehose.

Answer

Increase the source RDS instance storage to improve I/O.

Answer

Change the target data format from CSV to Parquet.

Question 14

A data engineer needs to ingest data from an external FTP server into S3 on a schedule. The FTP server is only accessible via VPN. Which AWS service is best suited for this task?

Accepted Answer

AWS Transfer Family. AWS Transfer Family supports FTP, FTPS, and SFTP protocols and can be integrated with a VPC using Elastic IPs or a VPC endpoint, enabling secure access to an FTP server reachable only via VPN. It automates the transfer of files from the external FTP server to Amazon S3 on a schedule without requiring custom code or infrastructure management.

Answer

AWS Snowcone

Answer

AWS Glue with a Python shell

Answer

AWS DataSync

Question 15

A data engineer is designing a near-real-time streaming pipeline to ingest clickstream data from a web application. The data must be enriched with user metadata from a DynamoDB table before being stored in S3. Which combination of AWS services should the engineer use? (Choose TWO.)

Accepted Answer

Amazon Kinesis Data Analytics for Apache Flink. Amazon Kinesis Data Streams (C) provides the low-latency, durable ingestion layer for the clickstream data, while Amazon Kinesis Data Analytics for Apache Flink (B) allows you to run a Flink application that can perform stream-to-stream joins with the DynamoDB user metadata in near-real time. The enriched output can then be written to S3 via a Kinesis Data Firehose delivery stream or directly from the Flink application.

Answer

Amazon Kinesis Data Firehose

Answer

AWS Lambda with DynamoDB Accelerator (DAX)

Answer

AWS Glue Streaming ETL