DEA-C01 Data Operations and Support Practice Test 34 — 15 Questions

Question 1

A data engineer is optimizing an AWS Glue ETL job that reads from Amazon S3 and writes to Amazon Redshift. The job currently uses a single large file and takes hours to complete. The engineer wants to improve performance by using partitioning and parallelism. Which TWO actions should the engineer take? (Choose two.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Convert the input data to a partitioned format, such as Parquet, and store it in S3 with a partition key.. To improve performance, the engineer should partition the input data and use a columnar format like Parquet, which enables Glue to read only necessary data and process partitions in parallel. Additionally, increasing the number of DPUs provides more compute resources to handle the parallel tasks. Together, these actions address both data layout and compute capacity, leading to faster job execution.

Answer

Configure the job to write output to a single file to reduce the number of S3 PUT requests.

Answer

Enable job bookmarks to track processed data and avoid reprocessing.

Answer

Use the Glue DynamicFrame instead of a DataFrame to enable automatic schema inference.

Question 2

A data engineer is using Amazon Managed Workflows for Apache Airflow (Amazon MWAA) to orchestrate a data pipeline. The engineer needs to ensure that the Airflow environment can access an Amazon S3 bucket to read and write data. The S3 bucket is in the same AWS account and Region as the MWAA environment. Which configuration is required to allow MWAA to access the S3 bucket?

Accepted Answer

Attach an IAM policy to the MWAA execution role that grants the necessary S3 permissions.. MWAA environments use an execution role that is assumed by the Airflow components. To grant access to S3, you attach an IAM policy to this role with the necessary S3 permissions. This is the standard and recommended approach. The execution role must also have a trust policy that allows MWAA to assume it, but that is created automatically when you create the environment.

Answer

Configure the S3 bucket policy to allow the MWAA service principal.

Answer

Enable public access on the S3 bucket and use the S3 REST API with presigned URLs.

Answer

Store S3 credentials in AWS Secrets Manager and reference them in the Airflow connection.

Question 3

A data engineer is using AWS Database Migration Service (AWS DMS) to replicate ongoing changes from an Amazon RDS for MySQL database to an Amazon S3 bucket in Parquet format. The replication task is configured with full load plus change data capture (CDC). After several hours, the engineer notices that the S3 bucket contains only the full load data and no incremental changes. Which action should the engineer take to ensure CDC changes are captured?

Accepted Answer

Enable binary logging on the source RDS for MySQL instance and set the binlog_format parameter to ROW.. For AWS DMS to perform ongoing replication from MySQL, the source must have binary logging enabled and binlog_format set to ROW. Without these settings, DMS cannot read the transaction log to capture changes. The engineer must enable binary logging and set the format to ROW, then restart the task. Other actions like scaling the instance or enabling Multi-AZ do not enable CDC.

Answer

Increase the DMS replication instance size to handle the change volume.

Answer

Enable Multi-AZ on the source RDS for MySQL instance.

Answer

Modify the DMS task to use a different target endpoint for CDC data.

Question 4

A data engineer is using AWS Glue to run a PySpark ETL job that processes millions of small JSON files stored in an Amazon S3 bucket. The job is experiencing high memory usage and failing with 'Container killed by YARN for exceeding memory limits'. The engineer wants to optimize the job to handle the data more efficiently without changing the source data format. Which solution will MOST effectively reduce memory usage and improve performance?

Accepted Answer

Use AWS Glue's groupFiles and groupSize options to combine small files into larger chunks during the read operation.. The groupFiles and groupSize options in AWS Glue are specifically designed to handle large numbers of small files by grouping them into larger chunks during read, which reduces the number of Spark partitions and memory overhead. This optimization directly targets the root cause of the memory failures and improves job efficiency without altering the source data format.

Answer

Convert the JSON files to Parquet format using an AWS Glue crawler before running the ETL job.

Answer

Increase the number of DPUs allocated to the AWS Glue job to provide more memory per executor.

Answer

Enable AWS Glue job bookmarks to track processed files and avoid reprocessing.

Question 5

A data engineer manages an AWS Glue job that processes JSON files from Amazon S3 and writes Parquet to another S3 location. The job intermittently fails with 'Unable to find catalog table' errors, even though the table exists in the Glue Data Catalog. The job's IAM role has full S3 access but only limited Glue permissions. Which action will resolve the failure with the LEAST privilege?

Accepted Answer

Add glue:GetTable and glue:GetDatabase permissions for the specific database and table to the job's IAM role.. The job fails because its IAM role cannot call Glue Data Catalog APIs to retrieve table metadata. The minimal fix is to grant glue:GetTable and glue:GetDatabase for the specific database and table. This resolves the error while adhering to least privilege, unlike broad managed policies or irrelevant S3 permissions.

Answer

Enable AWS Glue Data Catalog encryption and update the job's security configuration.

Answer

Attach the AWSGlueServiceRole managed policy to the job's IAM role.

Answer

Modify the job to use a different IAM role that has s3:GetObject permissions on the catalog bucket.

Question 6

A data engineer is managing an Amazon Redshift cluster that experiences performance degradation during peak query hours. The engineer needs to identify and resolve issues related to workload management (WLM). Which TWO actions should the engineer take to improve query performance? (Choose two.)

Accepted Answer

Enable short query acceleration (SQA) to prioritize short-running queries.. To improve query performance in Amazon Redshift during peak hours from a workload management perspective, the engineer should configure manual WLM queues with appropriate memory allocation and enable short query acceleration. Manual WLM allows fine-grained control over memory and concurrency, ensuring complex queries get necessary resources. SQA prioritizes short queries, reducing wait times. These actions directly address WLM-related bottlenecks.

Answer

Increase the number of nodes in the cluster to add more compute resources.

Answer

Set the WLM query slots to the maximum value for all queues to increase concurrency.

Answer

Disable concurrency scaling to prevent additional clusters from being added.

Question 7

A data engineer is managing an Amazon Redshift cluster that experiences performance degradation during peak query hours. The engineer notices that some queries are waiting in the queue for a long time, and the WLM (Workload Management) configuration is set to auto. The engineer wants to implement manual WLM to improve query throughput and ensure that short-running queries are not blocked by long-running ones. Which TWO actions should the engineer take to achieve this? (Choose two.)

Accepted Answer

Set the WLM timeout for the long-running query queue to automatically terminate queries that exceed a specified time.. Manual WLM with separate queues for short and long queries, along with WLM timeouts for long queries, directly addresses the issue of short queries being blocked. Separate queues provide resource isolation, while timeouts prevent long queries from consuming resources indefinitely. These two actions together improve throughput and ensure responsiveness during peak hours.

Answer

Use query monitoring rules (QMR) to log queries that exceed resource thresholds and send alerts.

Answer

Increase the number of nodes in the Redshift cluster to provide more memory and CPU resources.

Answer

Enable concurrency scaling to automatically add cluster capacity for bursts of queries.

Question 8

A data engineer is using AWS Step Functions to orchestrate an ETL workflow that includes an AWS Glue job, an Amazon EMR step, and an Amazon Redshift stored procedure. The workflow sometimes fails due to transient errors, and the engineer wants to implement a retry strategy that avoids duplicate data processing. Which TWO actions should the engineer take? (Choose two.)

Accepted Answer

Implement idempotency in the ETL tasks so that retries do not produce duplicate data.. To handle transient errors without duplicating data, the engineer should add Retry policies to Task states and ensure ETL tasks are idempotent. Retry policies automatically re-execute failed tasks, while idempotency guarantees that repeated executions do not create duplicate records. Together, they provide robust error handling and data integrity.

Answer

Configure the Step Functions execution to run in Express mode for faster retries.

Answer

Use a Catch block to redirect to a cleanup state that deletes partially written data before retrying the entire workflow.

Answer

Increase the timeout of the entire state machine to allow more time for tasks to complete.

Question 9

A data engineer is using AWS Step Functions to orchestrate a data pipeline that includes an AWS Glue job, an Amazon EMR step, and an Amazon Redshift stored procedure. The engineer needs to ensure that if the AWS Glue job fails, the pipeline retries the job up to three times before failing the entire execution. Which Step Functions state should the engineer use to implement this retry logic?

Accepted Answer

Task state with a Retry field configured with MaxAttempts set to 3.. AWS Step Functions Task states support a Retry field that allows you to specify retry behavior for a task, including the maximum number of attempts. By setting MaxAttempts to 3, the state machine will automatically retry the AWS Glue job up to three times if it fails. This is the standard and recommended way to implement retry logic for individual tasks.

Answer

Wait state with a Retry field to pause and then retry.

Answer

Map state with a Retry field to iterate over the Glue job.

Answer

Parallel state with a Catch field to handle failures.

Question 10

A data engineer maintains an AWS Glue job that reads JSON files from Amazon S3, applies a transform, and writes Parquet to a second bucket. The job's bookmark was enabled at creation, but each nightly run reprocesses all previously handled files, and downstream tables now contain duplicate rows. The job script has not been modified and the S3 prefix is unchanged. Which action will MOST directly resolve the duplicate processing?

Accepted Answer

Confirm the transformation_ctx parameter is passed to each source and sink call, then reset the job bookmark and rerun once to rebuild state.. AWS Glue job bookmarks persist per-source and per-sink state keyed by the transformation_ctx argument. If that context is missing or inconsistent, the job cannot determine which objects were already processed and re-reads the entire prefix, producing duplicates. Passing a stable transformation_ctx and resetting the bookmark to rebuild the state store fixes the incremental behavior without changing the transform logic.

Answer

Reprocess the prefix with a job that has job bookmarks disabled and rely on the S3 object LastModified timestamp to filter files.

Answer

Increase the number of AWS Glue DPUs allocated to the job so the run completes before the next scheduled trigger.

Answer

Change the job's output write mode to append and add a deduplication step that drops rows whose keys already exist.

Question 11

A data engineer is using AWS Step Functions to orchestrate a complex ETL workflow that includes multiple AWS Glue jobs, Amazon EMR steps, and AWS Lambda functions. The engineer notices that on rare occasions, the entire workflow fails due to a transient error in one of the Lambda functions. The engineer wants to make the workflow more resilient without changing the overall architecture. Which approach is the MOST effective?

Accepted Answer

Add a Retry state in the Step Functions state machine for the Lambda task with exponential backoff and a maximum number of attempts.. Adding a Retry state in Step Functions with exponential backoff and a maximum attempts limit is the most effective way to handle transient errors in Lambda functions. It automatically retries the failed task, increasing the likelihood of success without manual intervention, and is a standard resilience pattern in Step Functions.

Answer

Enable AWS X-Ray tracing for the Lambda function to identify the root cause of the errors.

Answer

Modify the Lambda function code to catch exceptions and return a success status to avoid workflow failure.

Answer

Configure the Lambda function to have a longer timeout and increase its memory size.

Question 12

A data engineer is using Amazon Athena to query data stored in Amazon S3. The engineer notices that queries are slow and scan large amounts of data. The data is stored in CSV format without compression. Which action should the engineer take to improve query performance and reduce cost?

Accepted Answer

Convert the data to Parquet format and compress it with Snappy.. Athena queries are optimized by using columnar formats like Parquet, which allow column pruning, and compression, which reduces data scanned. Converting from CSV to Parquet with Snappy compression will dramatically reduce the amount of data read, improving performance and lowering cost. Partitioning can further help but is secondary to format and compression.

Answer

Use Amazon Redshift Spectrum to query the S3 data instead of Athena.

Answer

Increase the number of Athena query executions by using a larger workgroup.

Answer

Partition the data by date and store it in CSV format.

Question 13

A data engineer runs an Apache Spark application on Amazon EMR that writes partitioned Parquet output to Amazon S3. A downstream AWS Glue crawler registers the table in the Data Catalog, but Athena queries return zero rows for partitions added by the most recent run, while older partitions query correctly. The S3 objects exist and are readable. Which action should the engineer take?

Accepted Answer

Run MSCK REPAIR TABLE or an ALTER TABLE ADD PARTITION statement against the table so the catalog discovers the newly written partitions.. Athena and Glue rely on partition metadata stored in the Data Catalog. Spark writes to S3 without automatically registering partitions unless the table is configured for it, so new prefixes remain invisible to the query planner and return no rows. Running MSCK REPAIR TABLE or adding partitions explicitly synchronizes the catalog with the S3 layout and makes the new data queryable.

Answer

Convert the table to use Athena partition projection with a date-based template that covers the new prefix layout.

Answer

Grant the crawler role s3:GetObject on the new partition prefixes so the crawler can inspect the files.

Answer

Increase the Athena query result reuse setting so previously cached partition metadata is invalidated.

Question 14

A data engineer needs to monitor an AWS Glue ETL job that runs daily. The job sometimes fails due to missing partitions in the Data Catalog. The engineer wants to receive an alert when the job fails. What is the MOST operationally efficient way to achieve this?

Accepted Answer

Use AWS Glue job event notifications via Amazon EventBridge to trigger an AWS Lambda function that sends an Amazon SNS notification.. Amazon EventBridge can capture AWS Glue job state change events and route them to targets like Lambda or SNS. This provides immediate, event-driven notifications on job failures without polling or custom log parsing. It is the most operationally efficient method for alerting on Glue job failures, including those due to missing partitions.

Answer

Enable AWS Glue job bookmarks and configure an Amazon CloudWatch Logs subscription filter to email the engineer on error patterns.

Answer

Create an Amazon CloudWatch alarm on the Glue job's 'glue.driver.aggregate.numFailedTasks' metric and notify an Amazon SNS topic.

Answer

Schedule a daily AWS Lambda function that calls the Glue GetJobRuns API and sends an email if the last run failed.

Question 15

A data engineer is managing an Amazon Redshift cluster and needs to load data from Amazon S3. The engineer uses the COPY command but encounters the error: 'S3ServiceException: Access Denied.' The Redshift cluster has an IAM role attached with permissions to access the S3 bucket. What is the most likely cause of this error?

Accepted Answer

The S3 bucket policy does not grant access to the IAM role used by Redshift.. The Access Denied error from S3 during a COPY operation typically indicates that the IAM role used by Redshift does not have sufficient permissions, or the S3 bucket policy does not allow the role. Even if the role has an S3 policy, an explicit deny or missing allow in the bucket policy can cause this. The engineer should verify both the IAM role's policies and the S3 bucket policy.

Answer

The IAM role attached to the Redshift cluster does not have the correct trust relationship with Redshift.

Answer

The Redshift cluster is not in the same AWS Region as the S3 bucket.

Answer

The COPY command is missing the CREDENTIALS parameter.