DEA-C01 Data Ingestion and Transformation Practice Test 16 — 15 Questions

Question 1

A company uses AWS Glue to run ETL jobs daily. The data is stored in S3 as Parquet files partitioned by date. Recently, jobs have failed with the error 'No such file or directory' for certain partitions. What is the MOST likely cause?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

A partition folder was deleted or not created by the upstream process.. The error 'No such file or directory' indicates that the Glue ETL job is attempting to read a specific S3 partition path that does not exist. Since the data is partitioned by date and the job runs daily, the most likely cause is that the upstream process failed to create or accidentally deleted the partition folder for that date. Glue's dynamic frame or Spark DataFrame will throw this error when it tries to list or read files from a missing prefix.

Answer

The schema has changed and Glue cannot parse the data.

Answer

The files are compressed with an unsupported codec.

Answer

The IAM role does not have s3:GetObject permission.

Question 2

A data engineering team needs to ingest streaming data from thousands of IoT devices and store it in Amazon S3 for batch processing. The data arrives at a rate of 10 MB/s, with occasional spikes up to 50 MB/s. The data must be processed in near real-time with minimal latency. Which AWS service should be used for ingestion?

Accepted Answer

Amazon Kinesis Data Streams. Amazon Kinesis Data Streams is designed for real-time streaming data ingestion at scale, handling throughput from megabytes to gigabytes per second with low latency. It can absorb the described 10 MB/s baseline and 50 MB/s spikes by sharding, and integrates directly with AWS Lambda or Kinesis Data Firehose to land data into Amazon S3 for batch processing.

Answer

Amazon DynamoDB Streams

Answer

Amazon SQS

Answer

Amazon S3

Question 3

A company runs a daily batch ETL job using AWS Glue. The job processes 500 GB of data from Amazon RDS to Amazon S3. The job currently uses a single DPU and takes 6 hours to complete. The team wants to reduce runtime to under 1 hour without increasing costs significantly. Which approach should they use?

Accepted Answer

Use AWS Glue Spark job with 100 workers.. AWS Glue Spark jobs can parallelize data processing across multiple workers, dramatically reducing runtime. With 100 workers, the job can process the 500 GB dataset in parallel, achieving sub-1-hour runtime while keeping costs relatively low since Glue charges per DPU-second and the total DPU-seconds may be similar to the original 6-hour single-DPU job.

Answer

Change the job type from Python to Spark.

Answer

Use multiple Glue jobs triggered sequentially.

Answer

Increase the RDS instance size to improve read throughput.

Question 4

A company needs to transfer 10 TB of historical data from an on-premises HDFS cluster to Amazon S3. The data is stored on a single 20 TB disk. The network link to AWS has a bandwidth of 1 Gbps. The transfer must be completed within 2 days. Which solution meets these requirements?

Accepted Answer

Use AWS Snowball Edge to transfer the data physically.. AWS Snowball Edge is the correct solution because it can physically transfer 10 TB of data from a single 20 TB disk within the 2-day window, bypassing the network bandwidth limitation. With a 1 Gbps link, the theoretical maximum transfer of 10 TB would take approximately 22.2 hours under ideal conditions, but real-world overhead, latency, and potential congestion make it unreliable to complete within 2 days. Snowball Edge provides a petabyte-scale physical transport solution that avoids network constraints entirely.

Answer

Use Amazon Kinesis Data Streams to stream data to S3.

Answer

Use AWS DMS to migrate data from HDFS to S3.

Answer

Use AWS CLI to copy data directly to S3 over the network.

Question 5

A data engineer notices that an AWS Glue job writing to Amazon S3 in Parquet format creates many small files (less than 1 MB each). This leads to poor query performance in Amazon Athena. What is the BEST way to reduce the number of output files?

Accepted Answer

Enable 'groupFiles' in the Glue job's S3 target configuration.. Enabling 'groupFiles' in the AWS Glue job's S3 target configuration instructs Glue to coalesce small files into larger ones (default target size ~128 MB) during the write phase. This directly reduces the number of small Parquet files written to S3, improving Athena query performance by minimizing S3 LIST and GET overhead.

Answer

Use 'coalesce(1)' at the end of the ETL script.

Answer

Use 'repartition(100)' to increase parallelism.

Answer

Configure an S3 lifecycle policy to delete small files.

Question 6

A company uses AWS Glue to process data from Amazon RDS MySQL into Amazon S3. The Glue job uses a JDBC connection and runs on a schedule. Recently, the job has been failing with a 'Communications link failure' error. The RDS instance is in a private subnet. Which troubleshooting step should the data engineer take FIRST?

Accepted Answer

Verify that the Glue job's VPC subnet and security group allow outbound traffic to RDS.. The 'Communications link failure' error typically indicates a network connectivity issue between the Glue job and the RDS instance. Since the RDS instance is in a private subnet, the Glue job must be configured with a VPC subnet and security group that allows outbound traffic to the RDS instance's security group on port 3306 (MySQL). Without this network path, the JDBC connection cannot be established, making verifying the VPC and security group configuration the first logical troubleshooting step.

Answer

Check the Glue job's DPU allocation; increase if too low.

Answer

Review the Glue job script for data type mismatches.

Answer

Increase the RDS instance's max_connections parameter.

Question 7

A data engineer needs to ingest real-time clickstream data from a website into Amazon S3 for analytics. The data arrives as JSON records, each under 1 KB. The engineer wants to use a serverless solution with automatic scaling and minimal operational overhead. Which AWS service should be used as the ingestion endpoint?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct choice because it is a fully managed, serverless service designed to ingest streaming data and automatically load it into Amazon S3 with no ongoing administration. It handles automatic scaling, converts incoming JSON records to formats like Parquet or ORC if needed, and can batch data into S3 based on time or size intervals, making it ideal for real-time clickstream ingestion with minimal operational overhead.

Answer

Amazon S3 with presigned URLs

Answer

Amazon Kinesis Data Analytics

Answer

AWS Lambda function behind an API Gateway

Question 8

A company uses AWS DMS to migrate data from an on-premises Oracle database to Amazon RDS for PostgreSQL. The migration is ongoing with continuous replication. The data engineer notices that some changes are not being captured in the target database. What is the MOST likely cause?

Accepted Answer

The source Oracle database is not configured to retain archived redo logs for a sufficient period.. In AWS DMS continuous replication (CDC), the service reads changes from the source Oracle database's archived redo logs. If the logs are rotated or deleted before DMS has a chance to process them, changes are lost. Option D is correct because insufficient retention of archived redo logs is the most likely cause of missing changes in the target.

Answer

The VPC peering connection between on-premises and AWS is down.

Answer

The DMS task's table mapping is incorrectly configured.

Answer

DMS is not publishing task logs to CloudWatch Logs.

Question 9

A company runs an AWS Glue ETL job that reads data from Amazon S3, transforms it, and writes back to S3 in a different partition structure. The job uses the 'spark.sql.shuffle.partitions' option set to 200. After the job completes, the output has many small files. The data engineer wants to minimize the number of output files while maintaining job performance. Which action should the engineer take?

Accepted Answer

Use 'coalesce(n)' with n based on target file size (e.g., 128 MB) before writing.. 'coalesce(n)' reduces the number of partitions without triggering a full shuffle, allowing you to control the number of output files based on a target file size (e.g., 128 MB). This minimizes small files while preserving job performance, as coalesce is a narrow transformation that avoids the overhead of a shuffle. In contrast, 'repartition(n)' would cause a full shuffle, degrading performance.

Answer

Enable S3 multipart upload for the Glue job.

Answer

Increase the 'spark.sql.shuffle.partitions' to 500.

Answer

Reduce the 'spark.sql.shuffle.partitions' to 50.

Question 10

A data engineer is designing a data ingestion pipeline to load JSON files from Amazon S3 into Amazon Redshift. Which TWO methods can be used to load the data efficiently?

Accepted Answer

Use the Redshift COPY command to load from S3.. The Redshift COPY command is specifically designed to efficiently load large datasets from Amazon S3 by automatically parallelizing the data across cluster nodes, leveraging the cluster's compute resources for high-throughput ingestion. It supports JSON data natively via the 'json' option, making it ideal for loading JSON files directly from S3 without intermediate transformations.

Answer

Use Amazon Kinesis Data Firehose to directly load into Redshift.

Answer

Use AWS DMS to replicate from S3 to Redshift.

Answer

Use individual INSERT statements in a loop.

Question 11

A company is building a data lake on Amazon S3. They need to ingest data from multiple sources, including relational databases, streaming data, and log files. Which THREE AWS services can be used to ingest data into the data lake?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is a fully managed service for streaming data ingestion that can capture, transform, and load streaming data into Amazon S3 in near real-time. It supports sources like Amazon CloudWatch Logs, AWS IoT, and custom producers via the Kinesis Agent, making it ideal for log files and streaming data.

Answer

Amazon Athena

Answer

Amazon Redshift Spectrum

Question 12

A data engineer is troubleshooting an AWS Glue job that reads from Amazon RDS MySQL and writes to Amazon S3. The job runs successfully but takes longer than expected. The engineer wants to optimize performance. Which THREE actions would improve job performance?

Accepted Answer

Increase the number of DPUs allocated to the Glue job.. Increasing the number of DPUs (Data Processing Units) allocated to the AWS Glue job provides more parallel processing capacity, allowing the job to process data faster. This is a direct way to improve performance when the job is CPU or memory-bound, as Glue distributes the workload across the allocated DPUs.

Answer

Use a single JDBC connection per partition.

Answer

Convert the output format from Parquet to CSV.

Question 13

Refer to the exhibit. A data engineer is creating an IAM policy for an application that sends data to a Kinesis stream and stores processed data in S3. The policy is attached to an IAM role used by an EC2 instance. The application fails to write to S3 with an access denied error. What is the cause?

Accepted Answer

The policy does not allow s3:ListBucket on the bucket.. The error occurs because the IAM policy grants s3:PutObject but not s3:ListBucket on the target S3 bucket. When the application writes to S3, the AWS SDK often performs a ListBucket operation first to verify bucket existence or to handle multipart uploads, and without s3:ListBucket permission, the request is denied with an access denied error.

Answer

The IAM role is not attached to the EC2 instance profile.

Answer

The policy does not allow kinesis:PutRecord on the stream.

Answer

The EC2 instance does not have an internet gateway to reach S3.

Question 14

Refer to the exhibit. A data engineer runs the AWS CLI command to describe a Glue job. The job is expected to process new data incrementally using job bookmarks. However, the job reprocesses all data every time it runs. What is the MOST likely reason?

Accepted Answer

The ETL script does not use the 'transformation_ctx' parameter in its DynamicFrame transformations.. AWS Glue job bookmarks rely on the `transformation_ctx` parameter to track state. Without it, Glue cannot identify which data has already been processed, causing the job to reprocess all data on every run. The `transformation_ctx` must be passed to each DynamicFrame transformation (e.g., `apply_mapping`, `filter`, `join`) to enable bookmark-based incremental processing.

Answer

The job bookmark option is set to 'job-bookmark-enable' but should be 'job-bookmark-disable'.

Answer

The job's MaxRetries is set to 0, which disables bookmarks.

Answer

The Glue job's command name is 'glueetl', which does not support job bookmarks.

Question 15

Refer to the exhibit. A data engineer is running an AWS Glue job that reads data from an S3 source. The job fails with the error shown. What is the MOST likely cause?

Accepted Answer

One of the source files is empty or corrupted.. The error message indicates that the Glue job encountered a 'NullPointerException' or similar parsing failure when reading from S3. This typically occurs when a source file is empty or corrupted, causing the Spark DataFrame reader to fail during schema inference or data parsing. AWS Glue jobs rely on Spark's ability to read files; an empty or malformed file triggers a runtime error because Spark cannot extract any records or infer a valid schema from it.

Answer

The IAM role does not have s3:GetObject permission.

Answer

The file is in JSON format but the schema expects Parquet.

Answer

The Glue job has insufficient memory allocated.