DEA-C01 Data Operations and Support Practice Test 13 — 15 Questions

Question 1

A company runs a data pipeline on AWS Glue that processes streaming data from Amazon Kinesis Data Streams and writes results to an Amazon Redshift cluster. The pipeline has been running smoothly, but recently the Glue job started failing with 'ResourceNotFoundException' for the Redshift table. What should the data engineer check first?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Verify that the target Redshift table exists and was not dropped or renamed.. The error indicates the table does not exist or was deleted. Option B is wrong because IAM role issues would cause Access Denied, not ResourceNotFoundException. Option C is wrong because network issues would cause timeout or connection refused. Option D is wrong because schema changes could cause type mismatch but not ResourceNotFoundException.

Answer

Ensure the Redshift table schema matches the Glue job output.

Answer

Check the IAM role permissions for the Glue job to access Redshift.

Answer

Review security group rules for the Redshift cluster.

Question 2

Which TWO AWS services can be used to schedule and orchestrate ETL workflows that involve multiple steps and dependencies? (Choose 2.)

Accepted Answer

AWS Step Functions. The correct answers are D (AWS Step Functions) and E (Amazon Managed Workflows for Apache Airflow). AWS Step Functions is a serverless orchestration service that lets you coordinate multiple AWS services into flexible, multi-step workflows. It is ideal for ETL pipelines that require sequential steps, branching, and error handling. Amazon MWAA is a managed service for Apache Airflow, an open-source platform used to programmatically author, schedule, and monitor workflows. MWAA simplifies setting up and operating Airflow, making it suitable for complex ETL orchestration with dependencies. In contrast, AWS Batch (A) is designed for batch computing jobs, not workflow orchestration. AWS Lambda (B) is for individual serverless functions, not multi-step orchestration. AWS Data Pipeline (C) is an older ETL orchestration service that is less flexible and less commonly used compared to Step Functions and MWAA.

Answer

AWS Batch

Answer

AWS Lambda

Answer

AWS Data Pipeline

Question 3

A data engineer is tasked with designing a disaster recovery solution for a data lake stored in Amazon S3. The data lake contains sensitive customer data that must be replicated to a different AWS Region. The engineer needs to ensure that all objects, including those with encryption using SSE-KMS, are replicated. Which solution meets the requirements?

Accepted Answer

Enable S3 Cross-Region Replication (CRR) with the appropriate KMS key and IAM role.. S3 Cross-Region Replication (CRR) can replicate objects encrypted with SSE-KMS if the appropriate KMS key is configured and the IAM role has the necessary permissions for encryption operations. Option A (S3 Batch Operations) is designed for one-time bulk actions, not ongoing replication. Option C (S3 Transfer Acceleration) only speeds up data transfer but does not provide replication. Option D (AWS CLI s3 sync) is a manual, scheduled copy process and does not offer automatic, continuous replication.

Answer

Use S3 Batch Operations to copy objects to the destination bucket.

Answer

Use S3 Transfer Acceleration to copy objects across regions.

Answer

Use the AWS CLI s3 sync command scheduled in a cron job.

Question 4

A data engineer is troubleshooting a failed AWS Glue ETL job that reads from and writes to the S3 bucket 'example-bucket'. The job's IAM role has the policy shown in the exhibit. The job fails with an Access Denied error when writing to a prefix 'output/'. Which permission is MISSING?

Accepted Answer

s3:DeleteObject. The IAM policy grants s3:GetObject and s3:PutObject on 'example-bucket/*', which includes the 'output/' prefix, so write access is sufficient. However, AWS Glue ETL jobs often create temporary files or need to clean up staging data, requiring s3:DeleteObject permission. Without it, the job may fail with an Access Denied error when attempting to delete objects during the write process or when cleaning up after a failure. Option D (s3:DeleteObject) is the missing permission.

Answer

s3:PutObjectAcl

Answer

s3:GetBucketAcl

Answer

s3:ListBucket on the output prefix

Question 5

A company runs a data warehouse on Amazon Redshift. The data engineer notices that some queries are running slowly. Upon reviewing the system tables, the engineer finds that the 'svv_table_info' shows high 'unsorted' percentage for several large tables. What is the MOST effective action to improve query performance?

Accepted Answer

Run the VACUUM command on the tables.. VACUUM sorts the data and reclaims space, improving query performance. Option A is wrong because ANALYZE updates statistics but does not sort. Option C is wrong because increasing the number of nodes may help but is not the most direct fix for unsorted data. Option D is wrong because changing distribution style would require recreating the table.

Answer

Run the ANALYZE command on the tables.

Answer

Change the distribution style of the tables to ALL.

Answer

Increase the number of nodes in the Redshift cluster.

Question 6

A data engineer is running an AWS Glue ETL job that converts CSV files to Parquet. The job fails with the error shown in the exhibit. The input files are about 500 MB each. The job uses 5 workers of type G.1X (16 GB memory each). What is the MOST likely cause?

Accepted Answer

The output Parquet file size is too large for the executor memory. The error indicates an out-of-memory (OOM) error during the write phase. When converting large CSV files to Parquet, Spark must buffer the output data in memory before writing. With 500 MB input files and 5 workers of G.1X (16 GB each), the executor memory is limited. If the output Parquet file or partition is too large, it may exceed the executor's memory, causing OOM. Option B (data skew) could also cause OOM, but typically during shuffle, not write. Options C and D are unrelated to write OOM issues.

Answer

The data is highly skewed causing a single partition to receive too much data

Answer

The Spark driver does not have enough memory to handle the schema inference

Answer

The input CSV files are corrupt or have inconsistent schema

Question 7

A data engineer is designing a data pipeline that ingests data from an on-premises database into Amazon S3 using AWS Database Migration Service (DMS). The data must be encrypted at rest in S3 using SSE-S3. The engineer also needs to track changes to the source database in real time. Which DMS configuration should the engineer use?

Accepted Answer

Use DMS with ongoing replication (change data capture) enabled.. DMS with ongoing replication (change data capture) enables real-time tracking of changes from the source database. Option A is incorrect because using a snapshot only captures data at a point in time, not real-time changes. Option C is incorrect because a full load task only loads existing data without capturing ongoing changes. Option D is incorrect because streaming to Amazon Kinesis is unnecessary; DMS CDC can directly replicate changes to S3. Encryption at rest in S3 with SSE-S3 is automatically supported by DMS when writing to S3.

Answer

Use DMS with a snapshot of the source database.

Answer

Use DMS with a full load task only.

Answer

Use DMS with a full load task and then stream to Amazon Kinesis.

Question 8

A data engineer is investigating why Amazon Athena queries on the 'my-data-lake' bucket are slow. The table is partitioned by year/month/day. The exhibit shows the objects in one partition. What is the MOST likely cause of poor query performance?

Accepted Answer

The files are too small, causing excessive read overhead. The exhibit shows tiny files (50 bytes), which cause excessive metadata overhead and read operations in Athena, leading to poor query performance. Option B (compression) is not indicated as the primary issue. Option C (partition columns) is likely appropriate given the partition structure. Option D (CSV format) is not necessarily the cause; while Parquet may improve performance, the main issue here is the file size, not the format.

Answer

The files are not compressed

Answer

The partition columns are not appropriately chosen

Answer

The data format is CSV instead of Parquet

Question 9

A company uses Amazon Athena to query data stored in an S3 bucket. The data is partitioned by year, month, day, and hour. The data engineer notices that queries are scanning a large amount of data even with a WHERE clause on the partition columns. What is the MOST likely cause?

Accepted Answer

The table does not have partitions defined in the AWS Glue Data Catalog.. If partitions are not defined in the table, Athena cannot perform partition pruning. Option A is wrong because too many partitions improve pruning, not hinder scanning. Option C is wrong because S3 storage class does not affect scanning. Option D is wrong because compressed files reduce scan size, not increase.

Answer

The data has too many partitions, causing overhead.

Answer

The S3 bucket uses the S3 Glacier storage class.

Answer

The data files are compressed with GZIP.

Question 10

A data engineering team notices that an Amazon Kinesis Data Stream is frequently exceeding its shard write throughput limit, causing throttling. The team needs a long-term solution to handle variable write traffic without manual intervention. Which action should the team take?

Accepted Answer

Implement a buffer using Amazon S3 and AWS Lambda that aggregates records and writes to Kinesis in batches.. Using Amazon S3 and AWS Lambda to buffer and batch records before writing to Kinesis smooths out traffic spikes, reducing throttling without manual intervention. Option A is incorrect because the Kinesis Client Library (KCL) is used by consumers, not producers; it cannot throttle write throughput. Option B is incorrect because manually increasing shards during peak hours requires manual intervention and is not a long-term automated solution. Option C is incorrect because Amazon Kinesis Data Firehose is designed to deliver streaming data to destinations like S3, Redshift, or Elasticsearch, not to buffer records before writing to a Kinesis stream.

Answer

Configure the Kinesis Client Library to throttle consumption.

Answer

Increase the number of shards manually during peak hours.

Answer

Use Amazon Kinesis Data Firehose to buffer records before delivery to the stream.

Question 11

A data engineer is monitoring an Amazon Redshift cluster and notices that the 'WLM query wait time' metric is consistently high during peak hours. The cluster uses automatic WLM. The engineer wants to reduce query wait times without changing the cluster size. Which action is MOST effective?

Accepted Answer

Enable concurrency scaling.. Enabling concurrency scaling (Option A) is the most effective action because it automatically adds transient cluster capacity during peak loads, allowing more queries to run concurrently without increasing wait times. This is specifically designed to reduce WLM query wait time. Option B (manual WLM) requires tuning and does not add capacity. Option C (increasing max queries per queue) could increase concurrency but may lead to resource contention and longer wait times if the cluster is already saturated. Option D (short query acceleration) prioritizes short queries, which does not address overall wait times for all queries. Therefore, A is correct.

Answer

Change WLM to manual mode and increase the number of queues.

Answer

Increase the maximum number of queries per queue.

Answer

Enable short query acceleration (SQA).

Question 12

A data pipeline uses AWS Glue ETL jobs to process data from Amazon RDS for MySQL to Amazon S3. Recently, the jobs have been failing with the error 'Communications link failure' during the connection phase. The RDS instance is in a private subnet, and the Glue job uses a VPC endpoint for S3. What is the most likely cause?

Accepted Answer

The Glue job does not have a network path to the RDS instance because it is not attached to the same VPC subnet.. The 'Communications link failure' error during connection indicates a network connectivity issue between the AWS Glue job and the RDS instance. Even if the Glue job uses a VPC endpoint for S3, that endpoint does not provide connectivity to RDS. To connect to an RDS instance in a private subnet, the Glue job must be attached to the same VPC (e.g., via an elastic network interface) or have a network path through VPC peering or VPN. Option A (max connections) would yield a 'too many connections' error, not a communications link failure. Option B (KMS permissions) would cause an access denied error, not a connection failure. Option C (JDBC driver incompatibility) would typically produce a 'No suitable driver' or driver class not found error. Therefore, the most likely cause is that the Glue job lacks a network path to the RDS instance.

Answer

The RDS database has reached the maximum number of connections.

Answer

The Glue job does not have IAM permissions to decrypt the RDS database using AWS KMS.

Answer

The JDBC driver used by Glue is incompatible with the MySQL version.

Question 13

A company runs a data pipeline that uses AWS Glue to process data from an Amazon DynamoDB table and write results to Amazon S3. The Glue job runs on a schedule every hour. Recently, the job started failing intermittently with 'ProvisionedThroughputExceededException' errors from DynamoDB. What is the BEST solution?

Accepted Answer

Implement exponential backoff and retries in the Glue job for DynamoDB operations.. Implementing exponential backoff and retries in the Glue job is a best practice to handle transient throttling errors from DynamoDB such as ProvisionedThroughputExceededException. This approach allows the job to automatically retry failed operations with increasing delays, reducing the likelihood of sustained failures without requiring changes to the DynamoDB table's provisioned capacity. Option A is incorrect because DynamoDB Accelerator (DAX) is an in-memory cache that can reduce read traffic for cached items, but it does not guarantee elimination of ProvisionedThroughputExceeded exceptions, especially for read operations that are not cached or when the underlying issue is read throughput limits. Option B is incorrect because running the job less frequently does not address the intermittent throttling; the job may still encounter the same error when it runs. Option D is incorrect because, although increasing read capacity units could directly address read throttling, it is not the most cost-effective or best practice for dealing with occasional throttling. The recommended approach is to first implement exponential backoff and retries, only increasing provisioned capacity if throttling persists. Moreover, the ProvisionedThroughputExceededException in this scenario is due to read throughput limits, and increasing write capacity would not help.

Answer

Use DynamoDB Accelerator (DAX) to reduce read latency.

Answer

Change the Glue job schedule to run every 2 hours.

Answer

Increase the read capacity units of the DynamoDB table.

Question 14

A company uses Amazon S3 to store raw data and AWS Lambda to process files as they arrive. The Lambda function sometimes times out when processing large files. The team wants to improve reliability and scalability. Which approach should the team take?

Accepted Answer

Use Amazon S3 event notifications to send events to an Amazon SQS queue, and then have Lambda poll the queue in batches.. Using S3 event notifications to send events to an SQS queue decouples the file upload from processing. Lambda can then poll the queue in batches, processing multiple events per invocation. This improves reliability by allowing retries and scalability by handling spikes in file arrivals without timing out. Option A is incorrect because AWS Batch is designed for long-running batch jobs, not event-driven processing triggered by S3 events. Option B is incorrect because SNS is push-based and can still overwhelm Lambda, leading to timeouts. Option C is incorrect because increasing Lambda timeout and memory only postpones the problem without addressing the root cause of scaling and reliability.

Answer

Replace Lambda with AWS Batch and use S3 event notifications to trigger the batch job.

Answer

Use Amazon S3 event notifications to send events to an Amazon SNS topic, which triggers Lambda.

Answer

Increase the Lambda function timeout to 15 minutes and memory to 3 GB.

Question 15

A data engineer runs an AWS Glue Crawler that updates a table in the AWS Glue Data Catalog. The table is used by Amazon Athena queries. After the crawler runs, some queries start failing with the error 'HIVE_CANNOT_OPEN_SPLIT'. What is the most likely cause?

Accepted Answer

The crawler updated the schema and the partition metadata is inconsistent with the actual data.. The error 'HIVE_CANNOT_OPEN_SPLIT' occurs when Athena attempts to read a partition but the metadata in the Glue Data Catalog does not match the actual data in S3. This is commonly caused by running an AWS Glue Crawler that updates the schema of a table (e.g., adding, removing, or changing column types) without updating the partition metadata accordingly. The crawler may update the table schema but leave existing partition metadata unchanged, leading to a mismatch. Option A correctly identifies this cause. Option B is incorrect because a permissions issue would result in an access denied error, not a split error. Option C is incorrect because while Athena has a limit on the number of partitions per table (currently 20,000), this limit would cause a different error or query failure, not 'HIVE_CANNOT_OPEN_SPLIT'. Option D is incorrect because concurrent queries can cause throttling or other issues but not this specific split error.

Answer

The crawler does not have IAM permissions to read the S3 location.

Answer

The crawler created too many partitions, exceeding the Athena limit.

Answer

There are concurrent queries accessing the same table.