DEA-C01 Data Operations and Support Practice Test 35 — 15 Questions

Question 1

A data engineer manages an AWS Glue ETL job that reads CSV files from Amazon S3 and writes Parquet to another S3 prefix. The job recently started failing with the error 'AnalysisException: Unable to infer schema for CSV.' The engineer confirms the S3 path contains files and the IAM role has s3:GetObject permissions. The job's script uses glueContext.create_dynamic_frame.from_catalog with a database and table name. What is the MOST likely cause?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The AWS Glue Data Catalog table's SerDe parameters are misconfigured, so the crawler did not correctly identify the CSV delimiter or header.. When an AWS Glue job uses create_dynamic_frame.from_catalog, it depends entirely on the AWS Glue Data Catalog table definition to determine the schema. If the table's SerDe parameters are incorrect—such as a wrong delimiter, missing header, or incorrect classification—the job cannot infer the schema and throws the AnalysisException. Verifying and repairing the catalog table resolves the failure.

Answer

The AWS Glue job's bookmarks are enabled, causing it to skip files that were already processed and leaving no data to infer schema from.

Answer

The IAM role lacks s3:ListBucket permission on the source bucket, so the job cannot list the CSV files.

Answer

The CSV files are compressed with gzip, and AWS Glue cannot infer schema from compressed files.

Question 2

A data engineer is using Amazon Athena to query Parquet data in Amazon S3. Queries are slow and scan more data than expected. The data is partitioned by year/month/day in S3, but the AWS Glue Data Catalog table has no partition metadata. Which action will improve query performance and reduce data scanned?

Accepted Answer

Run MSCK REPAIR TABLE or use AWS Glue Crawler to populate partition metadata in the Data Catalog.. Athena relies on the AWS Glue Data Catalog to know which partitions exist. If the table has no partition metadata, Athena treats the entire S3 prefix as one dataset and scans all files. Registering partitions via MSCK REPAIR TABLE or a Glue Crawler enables partition pruning, so queries only read the relevant partitions. This reduces both data scanned and query latency.

Answer

Enable Athena query result reuse and set a longer result retention period.

Answer

Increase the number of Athena workgroups and set a higher data usage control limit.

Answer

Convert the data to CSV and use a SerDe that supports compression.

Question 3

A data engineer is using AWS Glue job bookmarks to process incremental data from Amazon S3. The job reads from a partitioned S3 path and writes to Amazon Redshift. After a recent run, the engineer notices that some new partitions were not processed. The job bookmark state shows that the job has already processed up to a certain timestamp. What is the most likely reason for the missing partitions?

Accepted Answer

The new partitions were created with a timestamp earlier than the last processed timestamp.. Glue job bookmarks use timestamps or partition values to determine what data has already been processed. If new partitions have timestamps older than the last processed timestamp, they are considered already processed and skipped. This often happens with late-arriving data. The engineer should either adjust the bookmark to use partition-based tracking or handle late data separately.

Answer

The Glue job bookmark was not enabled for the S3 data source.

Answer

The Redshift cluster's copy command does not support partitioned data.

Answer

The Glue job's IAM role lacks permissions to read the new partitions.

Question 4

A data engineer is using AWS Step Functions to orchestrate a series of AWS Glue jobs. One of the Glue jobs occasionally fails due to transient network issues. The engineer wants the Step Function to retry the failed Glue job up to 3 times with exponential backoff before failing the entire workflow. Which Step Functions state configuration should be used?

Accepted Answer

Add a Retry field in the Task state with ErrorEquals matching the Glue job's error, MaxAttempts: 3, and BackoffRate: 2.0.. The Retry field in Step Functions Task states is specifically designed to handle transient errors by retrying the task with configurable attempts and backoff. Setting MaxAttempts to 3 and BackoffRate to 2.0 achieves up to three retries with exponential backoff. This is the standard, declarative approach and avoids custom orchestration logic. It also allows matching on specific error types, ensuring retries only occur for relevant failures.

Answer

Configure a Catch field in the Task state to transition to a Wait state, then loop back to the Glue job Task.

Answer

Set the Glue job's MaximumRetries parameter to 3 in the job definition.

Answer

Use a Map state to iterate over the Glue job three times with a Wait state in between.

Question 5

A data engineer is responsible for monitoring an AWS Glue ETL job that runs daily. The job reads data from an Amazon S3 bucket and writes to an Amazon Redshift table. The engineer wants to receive an alert if the job fails or if it takes longer than expected to complete. Which AWS service should the engineer use to set up these alerts with the LEAST operational overhead?

Accepted Answer

Amazon CloudWatch Alarms based on AWS Glue job metrics, with Amazon SNS notifications.. Amazon CloudWatch Alarms can monitor AWS Glue job metrics and trigger Amazon SNS notifications when thresholds are exceeded. This setup requires no custom code and provides timely alerts on job failures or prolonged execution, making it the most efficient solution with minimal operational overhead.

Answer

Amazon CloudWatch Events (EventBridge) to trigger an AWS Lambda function that sends an email via Amazon SES.

Answer

AWS Glue job bookmarks to track job progress and send notifications on failure.

Answer

AWS CloudTrail to log AWS Glue API calls and analyze logs for failures.

Question 6

A data engineer is using AWS Glue to run an ETL job that reads from an Amazon RDS for PostgreSQL database and writes to Amazon S3. The job is configured with a JDBC connection to RDS. The engineer notices that the job fails intermittently with a 'Connection timed out' error. The RDS instance is in a private subnet, and the Glue job has been configured with a VPC connection. Which action should the engineer take to resolve the timeout issue?

Accepted Answer

Ensure the Glue job's VPC connection has a security group that allows outbound traffic to the RDS instance's security group on the database port.. The intermittent connection timeout is likely caused by security group rules blocking traffic between the AWS Glue VPC connection and the RDS instance. The Glue job's ENI security group must allow outbound traffic to RDS, and the RDS security group must allow inbound traffic from the Glue security group on the database port. Ensuring these rules are correctly configured resolves the issue.

Answer

Enable the Glue job's VPC connection to use a NAT gateway for outbound traffic.

Answer

Increase the size of the Glue job's DPU capacity to handle more concurrent connections.

Answer

Configure the Glue job to use a public internet gateway to connect to RDS.

Question 7

A data engineer is using Amazon Kinesis Data Firehose to deliver streaming data to Amazon S3. The data is in JSON format, and the engineer needs to convert it to Parquet before storage to optimize Athena queries. The Firehose delivery stream is configured with an AWS Lambda function for record transformation. However, the transformed data is still in JSON format in S3. What is the likely cause?

Accepted Answer

The Firehose delivery stream is not configured with the 'Convert record format' option to convert JSON to Parquet using an AWS Glue table.. Kinesis Data Firehose can convert JSON to Parquet using the record format conversion feature, which relies on an AWS Glue table for schema. This option must be enabled in the delivery stream; otherwise, data remains in JSON even if a Lambda function transforms records. The Lambda function cannot perform the conversion itself, so enabling the Firehose setting is the correct fix.

Answer

The Lambda function is missing the necessary IAM permissions to write Parquet files to S3.

Answer

The Lambda function is not correctly converting the JSON to Parquet; it must output Parquet-formatted records.

Answer

The S3 bucket is configured with an S3 Lifecycle policy that converts objects to Parquet after a certain period.

Question 8

A data engineer is designing a pipeline that ingests streaming data into Amazon S3 using Amazon Kinesis Data Firehose. The data must be delivered to S3 in Parquet format and partitioned by date. The engineer needs to configure the Firehose delivery stream. Which two actions are required to meet these requirements? (Choose two.)

Accepted Answer

Enable record format conversion in the Firehose delivery stream and select Apache Parquet as the output format.. To deliver Parquet, Firehose must use record format conversion, which requires selecting Parquet and referencing a Glue Data Catalog table for the schema. To partition by date, Firehose must use dynamic partitioning with a JQ expression that extracts the date from each record and includes it in the S3 prefix. Together, these two configurations satisfy the format and partitioning requirements.

Answer

Enable Amazon CloudWatch Logs for the Firehose delivery stream.

Answer

Set the Firehose buffer size to 128 MB and buffer interval to 900 seconds.

Answer

Attach an IAM role to Firehose that allows s3:PutObject and glue:GetTable.

Question 9

A data engineer is using AWS Database Migration Service (AWS DMS) to migrate an on-premises Oracle database to Amazon Aurora PostgreSQL. The migration is ongoing, and the engineer notices that some transactions are not being replicated to the target. The DMS task is configured for full load plus change data capture (CDC). Which action should the engineer take to troubleshoot the missing transactions?

Accepted Answer

Check the DMS task's CloudWatch logs for errors related to unsupported data types or DDL changes.. When AWS DMS CDC skips transactions, it is often due to errors such as unsupported data types or DDL changes. These errors are logged in CloudWatch Logs. Reviewing the logs allows the engineer to identify the specific cause and take corrective action, such as excluding the problematic table or adjusting the task settings. This is the most direct troubleshooting step.

Answer

Enable AWS DMS validation to compare source and target rows and identify discrepancies.

Answer

Modify the DMS task to use 'Drop and create' target table preparation mode.

Answer

Increase the DMS replication instance size to handle the transaction volume.

Question 10

A data engineer manages an AWS Glue ETL job that reads CSV files from Amazon S3 and writes Parquet to another S3 prefix. The job recently started failing with the error 'AnalysisException: Unable to infer schema for CSV.' The engineer confirms the S3 path contains files and the IAM role has s3:GetObject permissions. The job's script uses glueContext.create_dynamic_frame.from_catalog with a database and table name. What is the MOST likely cause?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Modify the Redshift WLM configuration to increase concurrency or add a new queue for the Glue job.. The failure is caused by Redshift's workload management (WLM) queue being full, which leads to connection timeouts and resets when the AWS Glue job attempts to write data. Adjusting the WLM configuration to increase concurrency or create a dedicated queue for the Glue job reduces wait times and prevents the connection resets. This directly addresses the root cause without unnecessary changes to the Glue job or architecture.

Answer

Configure the Glue job to use the Redshift JDBC driver with a longer connection timeout and retry logic.

Answer

Increase the number of AWS Glue DPUs allocated to the job to speed up processing.

Answer

Switch the Glue job to write to Amazon S3 first, then use a Redshift COPY command to load the data.

Question 11

A data engineer is using AWS Step Functions to orchestrate a daily pipeline that runs several AWS Glue jobs in sequence. One Glue job intermittently fails due to a transient Amazon S3 503 error. The engineer wants the state machine to automatically retry only that Glue job up to three times with exponential backoff, without retrying the other jobs. What should the engineer do?

Accepted Answer

Add a Retry field to the Glue job task state with ErrorEquals set to States.TaskFailed, MaxAttempts set to 3, and an appropriate IntervalSeconds and BackoffRate.. Step Functions retry logic is declared inside the state definition, so attaching a Retry field to the Glue job task scopes retries to that task alone. Specifying MaxAttempts of 3 with IntervalSeconds and BackoffRate produces exponential backoff for transient failures. The other choices either apply retries at the wrong layer, change pipeline structure, or use services that cannot restart executions.

Answer

Set the Glue job's MaximumRetries property to 3 in the job definition, then add a Catch field in the state machine to handle final failures.

Answer

Configure an Amazon CloudWatch alarm on Glue job failures and use an Application Auto Scaling policy to restart the state machine execution.

Answer

Wrap the entire state machine in a Map state and set MaxConcurrency to 1 so that failures are retried automatically by Step Functions.

Question 12

A data engineer is using AWS Lake Formation to manage access to a data lake stored in Amazon S3. The engineer grants SELECT permission on a table in the AWS Glue Data Catalog to an IAM role used by an Amazon Athena user. However, the user still cannot query the table and receives an 'Access Denied' error. The engineer verifies that the IAM role has the necessary AWS Lake Formation permissions and that the S3 bucket policy allows access. What is the most likely cause of the issue?

Accepted Answer

The AWS Lake Formation data lake location is not registered.. The root cause is that the S3 data location is not registered with AWS Lake Formation. Even with Lake Formation table permissions, if the underlying S3 path is not registered, Lake Formation cannot manage access, and Athena queries fail with 'Access Denied'. Registering the location enables Lake Formation to control S3 permissions and grant access based on its policies.

Answer

The IAM role lacks permissions to the AWS Glue Data Catalog.

Answer

The S3 bucket policy does not grant access to the IAM role.

Answer

The IAM role requires an explicit DENY in the S3 bucket policy.

Question 13

A data engineer is using AWS Step Functions to orchestrate a daily ETL pipeline that includes an AWS Glue job, an Amazon EMR step, and an Amazon Redshift stored procedure. The pipeline occasionally fails with the error 'States.TaskFailed' from the Glue job, but the Glue job's own logs show that it completed successfully. The Step Functions execution history shows that the Glue job task timed out after 15 minutes, while the Glue job actually ran for 18 minutes. The Step Functions state machine uses the optimized Glue service integration with a TaskTimeout of 900 seconds. Which change will allow the pipeline to complete successfully without reducing the Glue job's runtime?

Accepted Answer

Increase the Step Functions TaskTimeout to a value greater than the Glue job's maximum expected runtime, such as 3600 seconds.. The Step Functions task failed because its TaskTimeout was shorter than the Glue job's actual runtime. The optimized Glue service integration waits for the job to complete, but if the task exceeds the TaskTimeout, Step Functions marks it as failed even though the Glue job continues. Setting the TaskTimeout to a value greater than the job's maximum expected runtime ensures the state machine waits for the job to finish successfully.

Answer

Change the Step Functions integration from the optimized Glue service integration to the AWS SDK integration with a longer timeout.

Answer

Add a Retry policy with a maximum of 3 attempts and an interval of 60 seconds to the Glue job task.

Answer

Reduce the Glue job's runtime by increasing the number of DPUs so it finishes within 15 minutes.

Question 14

A data engineer is using AWS Database Migration Service (AWS DMS) to migrate an on-premises Oracle database to Amazon Redshift. The migration uses a full load plus change data capture (CDC). During the CDC phase, the engineer notices that some updates are not being applied to the target Redshift tables. The DMS task logs show no errors. What is the MOST likely cause?

Accepted Answer

The source Oracle database is not configured for supplemental logging, so DMS cannot capture all column changes.. Oracle CDC in AWS DMS relies on supplemental logging to capture column-level changes. If supplemental logging is not enabled, the redo logs lack the necessary information for DMS to reconstruct updates, leading to missed changes without errors. Enabling supplemental logging at the database or table level resolves the issue.

Answer

The Redshift target table does not have a primary key defined, causing DMS to skip updates.

Answer

The DMS replication instance is undersized, causing it to drop change events under load.

Answer

The DMS task is configured with 'Limited LOB mode' and large LOB columns are being truncated, causing updates to be ignored.

Question 15

A data engineer is managing an Amazon Kinesis Data Firehose delivery stream that writes to an Amazon S3 bucket. The engineer notices that some records are being delivered to the S3 bucket with a prefix of 'errors/' instead of the intended 'data/' prefix. The Firehose stream is configured with an AWS Lambda function for data transformation, and the S3 bucket has a lifecycle policy that transitions objects to Glacier after 30 days. Which TWO actions should the engineer take to ensure that only successfully transformed records are delivered to the 'data/' prefix and that failed records are handled appropriately? (Choose two.)

Accepted Answer

Specify an error prefix in the Firehose stream's S3 destination configuration so that failed records are delivered to a separate prefix.. To ensure only successful records go to the 'data/' prefix, the Lambda function must return a 'result' of 'Ok' for successful transformations and 'Dropped' or 'ProcessingFailed' for failures. Additionally, configuring an error prefix in the Firehose S3 destination directs failed records to a separate 'errors/' prefix. Together, these actions segregate successful and failed records correctly.

Answer

Enable the 'S3 backup' setting in the Firehose stream to back up all incoming records to a separate S3 bucket before transformation.

Answer

Increase the Firehose stream's buffer size and buffer interval to reduce the number of failed records.

Answer

Configure the Firehose stream's S3 destination to use a dynamic prefix based on the record's result status, such as 'data/' for success and 'errors/' for failure.