DEA-C01 Data Ingestion and Transformation Practice Test 35 — 15 Questions

Question 1

A company runs a daily batch ETL job using AWS Glue. The job processes 500 GB of data from Amazon RDS to Amazon S3. The job currently uses a single DPU and takes 6 hours to complete. The team wants to reduce runtime to under 1 hour without increasing costs significantly. Which approach should they use?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use AWS Glue Spark job with 100 workers.. AWS Glue Spark jobs can parallelize data processing across multiple workers, dramatically reducing runtime. With 100 workers, the job can process the 500 GB dataset in parallel, achieving sub-1-hour runtime while keeping costs relatively low since Glue charges per DPU-second and the total DPU-seconds may be similar to the original 6-hour single-DPU job.

Answer

Change the job type from Python to Spark.

Answer

Use multiple Glue jobs triggered sequentially.

Answer

Increase the RDS instance size to improve read throughput.

Question 2

A company uses AWS Glue to process data from Amazon RDS MySQL into Amazon S3. The Glue job uses a JDBC connection and runs on a schedule. Recently, the job has been failing with a 'Communications link failure' error. The RDS instance is in a private subnet. Which troubleshooting step should the data engineer take FIRST?

Accepted Answer

Verify that the Glue job's VPC subnet and security group allow outbound traffic to RDS.. The 'Communications link failure' error typically indicates a network connectivity issue between the Glue job and the RDS instance. Since the RDS instance is in a private subnet, the Glue job must be configured with a VPC subnet and security group that allows outbound traffic to the RDS instance's security group on port 3306 (MySQL). Without this network path, the JDBC connection cannot be established, making verifying the VPC and security group configuration the first logical troubleshooting step.

Answer

Check the Glue job's DPU allocation; increase if too low.

Answer

Review the Glue job script for data type mismatches.

Answer

Increase the RDS instance's max_connections parameter.

Question 3

A company runs an AWS Glue ETL job that reads data from Amazon S3, transforms it, and writes back to S3 in a different partition structure. The job uses the 'spark.sql.shuffle.partitions' option set to 200. After the job completes, the output has many small files. The data engineer wants to minimize the number of output files while maintaining job performance. Which action should the engineer take?

Accepted Answer

Use 'coalesce(n)' with n based on target file size (e.g., 128 MB) before writing.. 'coalesce(n)' reduces the number of partitions without triggering a full shuffle, allowing you to control the number of output files based on a target file size (e.g., 128 MB). This minimizes small files while preserving job performance, as coalesce is a narrow transformation that avoids the overhead of a shuffle. In contrast, 'repartition(n)' would cause a full shuffle, degrading performance.

Answer

Enable S3 multipart upload for the Glue job.

Answer

Increase the 'spark.sql.shuffle.partitions' to 500.

Answer

Reduce the 'spark.sql.shuffle.partitions' to 50.

Question 4

A company is building a data lake on Amazon S3. They need to ingest data from multiple sources, including relational databases, streaming data, and log files. Which THREE AWS services can be used to ingest data into the data lake?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is a fully managed service for streaming data ingestion that can capture, transform, and load streaming data into Amazon S3 in near real-time. It supports sources like Amazon CloudWatch Logs, AWS IoT, and custom producers via the Kinesis Agent, making it ideal for log files and streaming data.

Answer

Amazon Athena

Answer

Amazon Redshift Spectrum

Question 5

A data engineer is troubleshooting an AWS Glue job that reads from Amazon RDS MySQL and writes to Amazon S3. The job runs successfully but takes longer than expected. The engineer wants to optimize performance. Which THREE actions would improve job performance?

Accepted Answer

Increase the number of DPUs allocated to the Glue job.. Increasing the number of DPUs (Data Processing Units) allocated to the AWS Glue job provides more parallel processing capacity, allowing the job to process data faster. This is a direct way to improve performance when the job is CPU or memory-bound, as Glue distributes the workload across the allocated DPUs.

Answer

Use a single JDBC connection per partition.

Answer

Convert the output format from Parquet to CSV.

Question 6

Refer to the exhibit. A data engineer runs the AWS CLI command to describe a Glue job. The job is expected to process new data incrementally using job bookmarks. However, the job reprocesses all data every time it runs. What is the MOST likely reason?

Accepted Answer

The ETL script does not use the 'transformation_ctx' parameter in its DynamicFrame transformations.. AWS Glue job bookmarks rely on the `transformation_ctx` parameter to track state. Without it, Glue cannot identify which data has already been processed, causing the job to reprocess all data on every run. The `transformation_ctx` must be passed to each DynamicFrame transformation (e.g., `apply_mapping`, `filter`, `join`) to enable bookmark-based incremental processing.

Answer

The job bookmark option is set to 'job-bookmark-enable' but should be 'job-bookmark-disable'.

Answer

The job's MaxRetries is set to 0, which disables bookmarks.

Answer

The Glue job's command name is 'glueetl', which does not support job bookmarks.

Question 7

A company uses AWS Glue ETL jobs to transform data in Amazon S3. The data arrives in JSON format but needs to be converted to Parquet for efficient querying. Which AWS Glue feature should be used to infer the schema and generate transformation code?

Accepted Answer

AWS Glue crawlers. AWS Glue crawlers are the correct feature because they automatically connect to data stores (like S3), infer the schema of JSON data by sampling it, and populate the AWS Glue Data Catalog with table definitions. This catalog schema can then be used by AWS Glue ETL jobs to generate transformation code (e.g., converting JSON to Parquet) without manual schema definition.

Answer

Amazon S3 Select

Answer

Amazon Athena

Answer

Amazon Kinesis Data Analytics

Question 8

A data engineer needs to capture change data capture (CDC) events from an Amazon RDS for PostgreSQL database and stream them to Amazon S3 in near real-time. Which AWS service should be used?

Accepted Answer

AWS Database Migration Service (AWS DMS). AWS DMS supports continuous replication from PostgreSQL source databases using logical replication slots to capture CDC events in near real-time. It can directly stream these changes to Amazon S3 as a target, making it the correct choice for this use case.

Answer

Amazon S3 Transfer Acceleration

Answer

Amazon Athena

Answer

Amazon Kinesis Data Streams

Question 9

A company uses AWS Glue to process data in Amazon S3. The Glue job fails with an error indicating that the partition keys in the catalog do not match the actual S3 partition structure. What is the most likely cause?

Accepted Answer

The Glue Data Catalog partition metadata is outdated after the S3 structure changed. The Glue Data Catalog stores partition metadata separately from the actual S3 partition layout. When the S3 partition structure changes (e.g., new partitions are added or existing ones are renamed) without updating the catalog, the Glue job reads stale partition metadata, leading to a mismatch error. The job fails because it expects partitions based on the catalog, not the live S3 structure.

Answer

The IAM role does not have permissions to read the S3 data

Answer

The data files are encrypted with SSE-KMS

Answer

The table name in the catalog is different from the one used in the job

Question 10

A data engineer is designing a streaming pipeline using Amazon Kinesis Data Streams with a shard count of 10. The incoming data rate is 1 MB/second. The consuming application uses the Kinesis Client Library (KCL) with a single worker. What is the most likely performance bottleneck?

Accepted Answer

The single KCL worker cannot process all shards in parallel. The Kinesis Client Library (KCL) uses a 1:1 mapping between shards and record processors by default. With 10 shards and only a single KCL worker, that worker must run all 10 record processors sequentially on a single host, creating a bottleneck. The worker cannot process records from multiple shards in parallel, so the throughput is limited by the single worker's processing capacity, not the stream's write capacity.

Answer

The Lambda function invoked by the stream has a cold start issue

Answer

The data stream has insufficient write capacity

Answer

The shard count is too low to handle the data rate

Question 11

A company needs to ingest data from multiple on-premises databases into Amazon S3 for analytics. The databases include Oracle, MySQL, and PostgreSQL. The data must be continuously replicated with minimal latency. Which AWS service should be used?

Accepted Answer

AWS Database Migration Service (AWS DMS). AWS DMS supports continuous replication (change data capture, CDC) from Oracle, MySQL, and PostgreSQL to S3 as a target, enabling near-real-time data ingestion with minimal latency. It handles schema conversion and can replicate ongoing changes without interrupting source databases, making it the correct choice for this use case.

Answer

Amazon Kinesis Data Streams

Answer

AWS Snowball

Answer

AWS Glue

Question 12

A data engineer is building a data pipeline that ingests data from Amazon S3 into Amazon Redshift. The data is in CSV format and includes a timestamp column. The pipeline should load only new data incrementally. Which approach is most efficient?

Accepted Answer

Use the COPY command with a manifest file that lists only the new S3 objects. Using the COPY command with a manifest file allows you to explicitly list only the new S3 objects to be loaded, enabling incremental loading without scanning or loading the entire bucket. This approach is efficient as it avoids the overhead of deduplication or full-bucket scans, and it leverages Redshift's native high-speed parallel ingestion from S3.

Answer

Use the COPY command to load the entire bucket and rely on Redshift to deduplicate

Answer

Use Amazon Redshift Spectrum to query the S3 data directly without loading

Answer

Use INSERT statements within a loop to load each new file

Question 13

Which TWO AWS services can be used to ingest streaming data into Amazon S3? (Choose two.)

Accepted Answer

Amazon Managed Streaming for Apache Kafka (Amazon MSK). Amazon Kinesis Data Firehose is the easiest way to reliably load streaming data into Amazon S3. It can capture, transform, and deliver streaming data to S3 destinations in near real-time with no code required. Amazon MSK (Managed Streaming for Apache Kafka) can also ingest streaming data into S3 by using Kafka Connect with an S3 sink connector, which writes data from Kafka topics directly to S3.

Answer

Amazon S3 Transfer Acceleration

Answer

Amazon Elastic Block Store (Amazon EBS)

Answer

AWS Snowball

Question 14

Which TWO practices improve the performance of AWS Glue ETL jobs? (Choose two.)

Accepted Answer

Use pushdown predicates to filter data at the source. Pushdown predicates (Option A) improve AWS Glue ETL performance by filtering data at the source before it is read into the job. This reduces the volume of data transferred and processed, which is especially effective when using formats like Parquet or ORC that support predicate pushdown natively. By applying filters early, Glue avoids scanning unnecessary partitions or rows, leading to faster execution and lower costs.

Answer

Increase the number of DPUs to the maximum allowed

Answer

Use the smallest possible file size for input data

Answer

Enable AWS Glue job metrics and debug logging

Question 15

Which THREE factors should be considered when choosing between Amazon Kinesis Data Streams and Amazon Kinesis Data Firehose for real-time data ingestion? (Choose three.)

Accepted Answer

The need for a fully managed delivery destination. Amazon Kinesis Data Firehose is a fully managed service that automatically delivers streaming data to destinations like Amazon S3, Amazon Redshift, Amazon OpenSearch Service, and Splunk, making it ideal when you need a fully managed delivery destination without managing the ingestion pipeline. In contrast, Amazon Kinesis Data Streams requires you to build and manage consumers to process and deliver data, so if you need a fully managed destination, Firehose is the correct choice.

Answer

The ability to compress data before storage

Answer

The maximum throughput supported per shard