DEA-C01 Data Ingestion and Transformation Practice Test 13 — 15 Questions

Question 1

A company uses AWS Glue DataBrew to clean and transform data for analytics. The source data is in Parquet format in Amazon S3. The transformation includes filtering rows and adding calculated columns. What is the MOST cost-effective way to run these transformations on a schedule?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Create a Glue DataBrew recipe and schedule the job using a cron expression. AWS Glue DataBrew is purpose-built for visual data preparation, and scheduling a DataBrew recipe job with a cron expression directly meets the requirement to run filtering and column calculations on Parquet data in S3 without writing code. This is the most cost-effective approach as it avoids provisioning or managing compute resources beyond the serverless DataBrew job runs.

Answer

Use Amazon EMR with Spark

Answer

Create an AWS Lambda function triggered by S3 events

Answer

Use AWS Glue ETL with PySpark

Question 2

A company uses Amazon Kinesis Data Analytics for Apache Flink to process streaming data. The application reads from a Kinesis stream with 10 shards and writes to an S3 bucket. The application is experiencing high latency. Analysis shows that the application is not keeping up with the incoming data rate. Which action would MOST effectively reduce latency?

Accepted Answer

Increase the Parallelism of the Flink application. The high latency is caused by the Flink application not keeping up with the incoming data rate, which indicates a processing bottleneck within the application itself. Increasing the Parallelism of the Flink application (Option B) directly increases the number of parallel subtasks that can process data concurrently, improving throughput and reducing latency. This is the most effective action because it addresses the root cause—insufficient compute resources for stream processing—without changing the source or sink configuration.

Answer

Increase the number of shards in the Kinesis stream

Answer

Enable exactly-once delivery to S3

Answer

Use a larger Kinesis Data Analytics application (increase KPU)

Question 3

A company uses AWS Lake Formation to manage data lake permissions. A data engineer needs to grant an IAM role 'Read' access to a specific database and all its tables in the Data Catalog. What is the MOST efficient way to achieve this?

Accepted Answer

Grant 'Select' on the database and select 'Include' to apply to all tables. Lake Formation allows granting 'Select' permission on a database with the 'Include' option, which automatically applies the same permission to all current and future tables in that database. This is the most efficient way to grant read access to an IAM role for a specific database and all its tables in the Data Catalog, as it avoids manual per-table grants.

Answer

Grant 'Super' permission on the Data Catalog

Answer

Add the IAM role to the Lake Formation administrators group

Answer

Grant 'Describe' on the database and 'Select' on each table individually

Question 4

A company uses AWS Glue ETL to transform data from Amazon RDS for MySQL to Amazon S3. The Glue job reads from a JDBC connection. The job runs once daily and processes all records, but the data volume is growing. Which change would improve performance and reduce costs?

Accepted Answer

Enable Glue job bookmarking and set the job to process only new data. Enabling Glue job bookmarking allows the job to process only new or changed data since the last run, rather than reprocessing the entire dataset. This reduces both the data volume read from the JDBC source and the transformation time, directly improving performance and lowering costs by minimizing DPU usage.

Answer

Increase the number of DPUs for the Glue job

Answer

Switch to a Glue Python shell job

Answer

Use a higher JDBC fetch size

Question 5

Which TWO options are valid methods to ingest on-premises relational database data into Amazon S3 for analytics? (Choose 2.)

Accepted Answer

AWS Glue ETL job with JDBC connection to source. AWS Glue ETL jobs can connect to on-premises relational databases via JDBC, extract data, and write it directly to Amazon S3 in formats like Parquet or ORC. This is a fully managed, serverless approach suitable for batch ingestion and transformation of structured data for analytics.

Answer

AWS Snowball Edge

Answer

Amazon Kinesis Data Streams with Direct Put

Answer

Amazon AppFlow

Question 6

Which TWO AWS services can be used to transform data in transit during ingestion? (Choose 2.)

Accepted Answer

Amazon Kinesis Data Firehose with Lambda transformation. Amazon Kinesis Data Firehose can invoke an AWS Lambda function to transform streaming data in real time before delivering it to a destination, making it suitable for transforming data in transit during ingestion. AWS Glue ETL can be used for both batch and streaming transformations; specifically, AWS Glue supports streaming ETL jobs that can transform data in transit as it is ingested from sources like Amazon MSK or Kinesis Data Streams. Therefore, both services can transform data during ingestion, albeit with different use cases and latency characteristics.

Answer

Amazon S3 Transfer Acceleration

Answer

Amazon Athena

Answer

AWS Data Pipeline

Question 7

A company uses Amazon Kinesis Data Streams to ingest clickstream data from a website. The data is then consumed by a custom application for real-time analytics. Recently, the application has been experiencing high latency. The operations team suspects the shard count is insufficient. How should the team increase the shard count of the existing stream?

Accepted Answer

Use the UpdateShardCount API to increase the shard count for the stream.. The UpdateShardCount API is the correct method to increase the shard count of an existing Kinesis Data Stream without data loss or downtime. It allows you to specify a target shard count, and Kinesis automatically splits shards to achieve that count, redistributing the hash key range across the new shards. This directly addresses the high latency caused by insufficient shard count by increasing the stream's throughput capacity.

Answer

Delete the existing stream and create a new one with a higher shard count.

Answer

Manually split a shard using the SplitShard API on each existing shard.

Answer

Modify the PutRecord calls to include a new shard key that distributes data across more shards.

Question 8

A data engineer needs to ingest data from an on-premises Oracle database into Amazon S3 on a daily basis. The data volume is approximately 500 GB per day. The source database is behind a firewall that does not allow direct internet access. Which service should the engineer use to transfer the data securely?

Accepted Answer

AWS DataSync with a network path through AWS Direct Connect or VPN.. AWS DataSync can transfer data from on-premises storage to AWS over a network path established through AWS Direct Connect or a VPN, which is necessary because the source database is behind a firewall that blocks direct internet access. DataSync automates the movement of large datasets (up to 500 GB daily) and integrates with Oracle databases via a supported agent, handling incremental transfers efficiently. This makes it the correct choice for secure, scheduled, and high-volume data ingestion into Amazon S3.

Answer

AWS Database Migration Service (AWS DMS) with ongoing replication from Oracle to S3.

Answer

Amazon S3 Transfer Acceleration with a public endpoint.

Answer

AWS Snowball Edge device for daily transfers.

Question 9

A company uses AWS Glue to transform data stored in Amazon S3. During a run, the job fails with a 'OutOfMemoryError' in the Spark executor. The job processes 2 TB of parquet files using 10 DPUs. The data is evenly distributed across partitions. Which action would MOST likely resolve the issue without impacting the job logic?

Accepted Answer

Increase the number of DPUs allocated to the Glue job.. The OutOfMemoryError in the Spark executor indicates that the available memory per executor is insufficient for the data being processed. Increasing the number of DPUs allocated to the Glue job increases the total memory and compute resources available, allowing Spark to handle the 2 TB dataset without changing the job logic.

Answer

Enable S3 request rate increase to speed up data reading.

Answer

Repartition the data to a larger number of partitions.

Answer

Change the input format from Parquet to Snappy-compressed CSV.

Question 10

A company is designing a data ingestion pipeline for real-time IoT sensor data. The data volume peaks at 10,000 messages per second. The pipeline must process messages in order per sensor and persist raw data to Amazon S3 for archival. Which TWO services should be used together to meet these requirements? (Choose TWO.)

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Streams (KDS) provides the ordered, real-time ingestion layer required for per-sensor message ordering, as it partitions data by a partition key (e.g., sensor ID) and guarantees order within a shard. Amazon Kinesis Data Firehose then reliably reads from the KDS stream and delivers the raw data to Amazon S3 for archival, handling buffering, compression, and automatic retries without requiring custom code.

Answer

AWS Lambda

Answer

Amazon AppFlow

Answer

Amazon Simple Queue Service (Amazon SQS)

Question 11

A company uses Amazon Kinesis Data Analytics for Apache Flink to process streaming data. The Flink application reads from a Kinesis Data Streams source, performs aggregations, and writes results to Amazon S3. The application is experiencing high checkpoint failures, and the processing lag is increasing. The data volume is 50 MB/s with an average record size of 1 KB. Which TWO actions would improve checkpoint reliability and reduce lag? (Choose TWO.)

Accepted Answer

Increase the checkpoint interval in the Flink configuration.. Increasing the checkpoint interval (Option D) reduces the frequency of checkpoint operations, which decreases the overhead on the Flink application and allows it to dedicate more resources to processing data, thereby reducing lag. This is especially effective when checkpoint failures are caused by the system being unable to complete checkpoints within the current interval due to high throughput (50 MB/s).

Answer

Decrease the checkpoint interval to complete checkpoints faster.

Answer

Replace the S3 sink with Kinesis Data Firehose.

Answer

Decrease the parallelism of the Flink application.

Question 12

Refer to the exhibit. A data engineer is attaching this IAM policy to an IAM role used by an AWS Glue job. The job reads from a Kinesis Data Streams stream and writes transformed data to an S3 bucket. When the job runs, it fails with an AccessDenied error for the Kinesis stream. What is the MOST likely cause?

Accepted Answer

The IAM policy is missing the 'kinesis:DescribeStream' action.. When an AWS Glue job reads from Kinesis Data Streams, it requires the `kinesis:DescribeStream` action to retrieve stream metadata such as shard IDs and the stream's current state. Without this permission, the Glue job cannot discover the stream's shards and fails with an AccessDenied error, even if other Kinesis actions like `GetRecords` are allowed. The error occurs because the IAM policy attached to the Glue role is missing this prerequisite action.

Answer

The stream ARN in the policy is incorrect.

Answer

The Glue job does not have permissions to call 'kinesis:PutRecord'.

Answer

The S3 bucket policy blocks the PutObject action from the Glue role.

Question 13

Refer to the exhibit. A company uses S3 Event Notifications to trigger an AWS Lambda function whenever a new object is uploaded to an S3 bucket. The Lambda function processes the file and moves it to a different bucket. Recently, the function has been failing intermittently. The engineer checks the Lambda CloudWatch logs and sees the above event. What is the MOST likely cause of the intermittent failures?

Accepted Answer

The S3 event notifications are not guaranteed to be delivered exactly once, causing duplicate processing.. S3 Event Notifications are asynchronous and may be duplicated or delivered out of order, causing race conditions. Option A is wrong because the event has all required fields. Option B is wrong because the bucket name is valid. Option D is wrong because the event format is correct.

Answer

The event JSON is malformed; the 'EventSource' should be 's3.amazonaws.com'.

Answer

The S3 bucket name contains a hyphen, which is not allowed.

Answer

The event is missing the 'object:versionId' field.

Question 14

A company uses AWS Glue to run ETL jobs that process data from Amazon RDS to Amazon S3. The jobs run nightly and take 3 hours to complete. The data volume is growing by 20% each month. The engineer needs to reduce job runtime and cost. The source RDS is a db.r5.large instance. Which approach would be MOST effective?

Accepted Answer

Increase the number of Glue workers and choose a G.1X or G.2X worker type.. Increasing the number of Glue workers and choosing a G.1X or G.2X worker type directly increases parallelism and provides more memory per worker, reducing job runtime at a manageable cost increase. Option A is wrong; reducing DPUs would increase runtime, not reduce it. Option C is wrong because a read replica does not improve Glue processing speed; the bottleneck is Glue's processing capacity, not the source database's read capacity. Option D is wrong because S3 Transfer Acceleration improves upload speed to S3, not Glue job processing.

Answer

Reduce the number of DPUs to lower cost and accept longer runtime.

Answer

Create a read replica of the RDS instance and point the Glue job to the replica.

Answer

Enable S3 Transfer Acceleration on the destination bucket.

Question 15

A company wants to ingest streaming data from thousands of IoT devices into AWS for real-time processing. Each device sends JSON payloads of about 2 KB at a rate of 1 message per second. The data must be processed with a durable, ordered stream per device. Which service should the company use as the ingestion layer?

Accepted Answer

Amazon Kinesis Data Streams.. Amazon Kinesis Data Streams is the correct choice because it provides durable, ordered stream processing per shard, which can be partitioned by device ID to maintain message order for each device. It supports real-time ingestion from thousands of IoT devices at 2 KB per message per second, with a retention period of up to 365 days and the ability to reprocess data via multiple consumers.

Answer

Amazon Simple Queue Service (Amazon SQS) with a FIFO queue.

Answer

Amazon Simple Notification Service (Amazon SNS) with a Lambda subscriber.

Answer

Amazon Kinesis Data Firehose with Direct Put.