DEA-C01 Data Operations and Support Practice Test 33 — 15 Questions

Question 1

A data engineer is using AWS Step Functions to orchestrate a daily ETL workflow that includes an AWS Glue job, an Amazon EMR step, and an Amazon Redshift stored procedure. The workflow occasionally fails and the engineer needs to troubleshoot and recover. Which TWO actions should the engineer take to identify the failure and resume from the failed step? (Choose two.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use the Step Functions 'Redrive' capability to restart the execution from the failed state after fixing the underlying issue.. To troubleshoot a failed Step Functions execution, enabling logging and reviewing execution history provides the necessary error details. Once the root cause is fixed, the Redrive feature allows resuming from the failed state without re-running successful steps. These two actions together enable efficient diagnosis and recovery.

Answer

Delete the state machine and recreate it with a retry policy on every state.

Answer

Configure the state machine to use an Amazon SQS dead-letter queue for failed states and reprocess messages manually.

Answer

Manually trigger the AWS Glue job and Amazon EMR step from the console, then restart the Step Functions execution from the beginning.

Question 2

A data engineer is using Amazon Managed Workflows for Apache Airflow (MWAA) to orchestrate a data pipeline. The pipeline includes a task that runs an AWS Glue job. The engineer notices that the Glue job occasionally fails due to transient issues, and the Airflow task fails immediately without retrying. The engineer wants to configure the Airflow task to retry the Glue job up to 2 times with a 5-minute delay between retries. Which configuration in the Airflow DAG should the engineer use?

Accepted Answer

Set retries=2 and retry_delay=timedelta(minutes=5) in the default_args dictionary.. In Amazon MWAA, retry behavior for tasks is controlled by Airflow's built-in retry parameters. Setting retries and retry_delay in default_args applies to all tasks, including those running Glue jobs. This ensures that transient failures are retried with the specified delay, improving pipeline resilience.

Answer

Set max_retries=2 and retry_interval=300 in the Glue job's default arguments.

Answer

Use the retry_exponential_backoff=True parameter in the DAG definition.

Answer

Configure the Airflow task to use the GlueJobOperator with retry_limit=2 and retry_delay=300.

Question 3

A data engineer is using Amazon Kinesis Data Streams to ingest clickstream data. The stream has 10 shards and each record is 50 KB. The engineer notices that the PutRecords API is frequently returning ProvisionedThroughputExceededException errors, even though the total incoming data rate is below the stream's overall capacity. What is the MOST likely cause?

Accepted Answer

The partition key is not uniformly distributed across shards, causing hot shards.. ProvisionedThroughputExceededException occurs when a specific shard exceeds its 1 MB/s or 1000 records/s limit. If total incoming rate is below stream capacity but partition keys are not uniformly distributed, some shards become hot and throttle. Using a well-distributed partition key or resharding can mitigate this.

Answer

The stream's retention period is set too low, causing data loss.

Answer

The PutRecords API call is using too many records per request, exceeding the 500-record limit.

Answer

The stream's shards are not encrypted, causing throttling.

Question 4

A data engineer is using AWS Step Functions to orchestrate a daily ETL workflow that includes an AWS Glue job, an Amazon EMR step, and an Amazon Redshift stored procedure. The workflow occasionally fails and the engineer needs to troubleshoot and recover. Which TWO actions should the engineer take to identify the failure and resume from the failed step? (Choose two.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Add a Retry field to the state that invokes the Glue job.. The Retry field in Step Functions is specifically designed to handle transient errors by automatically retrying a state. It allows you to specify the error types to retry (e.g., States.TaskFailed), the maximum number of attempts, the interval between retries, and a backoff rate. This is the standard way to implement retry logic for a Glue job invocation within a state machine.

Answer

Configure a Catch field to transition to a retry state on failure.

Answer

Use a Parallel state to run the Glue job multiple times concurrently.

Answer

Increase the Glue job's timeout and memory allocation.

Question 5

A data engineer manages an AWS Glue ETL job that reads from an Amazon S3 bucket and writes to an Amazon Redshift table. The job runs daily and recently started failing with the error 'Unable to find a suitable security group for the connection'. The Glue connection is configured with a VPC, subnet, and security group. The engineer verifies that the IAM role has the necessary permissions and the S3 bucket is accessible. What is the most likely cause of this error?

Accepted Answer

The security group specified in the Glue connection does not exist or is not in the same VPC as the subnet.. The error 'Unable to find a suitable security group for the connection' occurs when the security group specified in the Glue connection is invalid for the subnet. This typically happens if the security group does not exist, is in a different VPC, or is not associated with the subnet's VPC. Ensuring the security group and subnet are in the same VPC and that the security group exists resolves the issue.

Answer

The security group attached to the Glue connection does not allow outbound traffic to the Redshift cluster on the required port.

Answer

The IAM role used by the Glue job lacks permissions to describe security groups in the VPC.

Answer

The Glue connection is associated with a subnet that does not have a route to the Redshift cluster.

Question 6

A data engineer is using Amazon Kinesis Data Firehose to deliver streaming data to an Amazon S3 bucket. The engineer notices that some records are being delivered to S3 with a delay of several minutes, and sometimes records are missing. The Firehose stream is configured with a buffer size of 5 MB and a buffer interval of 300 seconds. The engineer wants to reduce latency and ensure all records are delivered. Which action should the engineer take?

Accepted Answer

Reduce the buffer interval to 60 seconds and enable Amazon CloudWatch Logs for error logging.. The buffer interval determines how long Firehose waits before delivering data. A 300-second interval means records can be delayed up to 5 minutes. Reducing it to 60 seconds lowers latency. Missing records often indicate delivery errors, which can be diagnosed by enabling CloudWatch Logs for the Firehose stream. Firehose logs errors related to S3 delivery, Lambda processing, and format conversion, helping identify and fix issues.

Answer

Increase the buffer size to 128 MB to allow more data to be batched.

Answer

Enable Amazon S3 versioning on the destination bucket.

Answer

Configure the Firehose stream to use AWS Lambda for data transformation.

Question 7

A data engineer is using Amazon Athena to query data stored in Amazon S3. The data is partitioned by date, but the engineer notices that queries are scanning the entire bucket instead of only the relevant partitions. The table is defined in the AWS Glue Data Catalog. Which action should the engineer take to ensure that Athena only scans the necessary partitions?

Accepted Answer

Run the `MSCK REPAIR TABLE` command in Athena to update the partition metadata in the AWS Glue Data Catalog.. When Athena scans the entire bucket despite partitioning, it is often because the partition metadata in the AWS Glue Data Catalog is not up to date. Running `MSCK REPAIR TABLE` in Athena adds missing partitions to the catalog, allowing Athena to prune partitions and scan only the relevant data.

Answer

Increase the query timeout setting for the Athena workgroup to allow more time for scanning.

Answer

Convert the data to a columnar format such as Parquet and enable predicate pushdown.

Answer

Use AWS Glue crawlers to re-crawl the data and update the table schema.

Question 8

A data engineer needs to grant an AWS Lambda function permission to read objects from a specific Amazon S3 bucket. The Lambda function assumes an IAM role. Which policy should the engineer attach to the IAM role to allow the Lambda function to read objects from the bucket?

Accepted Answer

An IAM policy with s3:GetObject and s3:ListBucket actions, with the resource set to the bucket ARN and objects ARN.. To read objects from S3, the Lambda function requires s3:GetObject on the object ARN and s3:ListBucket on the bucket ARN. This combination allows listing the bucket and retrieving objects. Policies that omit the object ARN for GetObject or include unnecessary write permissions are incorrect. The correct policy follows the principle of least privilege.

Answer

An IAM policy with s3:PutObject and s3:GetObject actions, with resource set to the bucket ARN and objects ARN.

Answer

An IAM policy with s3:ListBucket action and resource set to the objects ARN only.

Answer

An IAM policy with s3:GetObject action and resource set to the bucket ARN only.

Question 9

A data engineer is using AWS Step Functions to orchestrate a daily ETL workflow that includes an AWS Glue job, an Amazon EMR step, and an Amazon Redshift stored procedure. The workflow occasionally fails and the engineer needs to troubleshoot and recover. Which TWO actions should the engineer take to identify the failure and resume from the failed step? (Choose two.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The Step Functions task has a TimeoutSeconds value set to 900, overriding the state machine timeout.. The most likely cause is that the Step Functions task has a TimeoutSeconds parameter set to 900 seconds (15 minutes). This parameter overrides the state machine's timeout for that specific task. To allow the Glue job to run for its full duration, the engineer should increase the task timeout to a value greater than the expected job runtime, or remove it to rely on the state machine's timeout.

Answer

The AWS Glue job is running in a VPC without a NAT gateway, causing network timeouts.

Answer

The Step Functions task is using the optimized integration for AWS Glue, which has a maximum timeout of 15 minutes.

Answer

The AWS Glue job is not sending a heartbeat to Step Functions, so the task times out.

Question 10

A data engineer manages an Amazon Kinesis Data Stream with multiple shards. The stream is experiencing high throughput, and the engineer notices that some shards are throttling while others are underutilized. The engineer needs to redistribute the data evenly across shards to avoid throttling. Which action should the engineer take?

Accepted Answer

Modify the partition key strategy in the producer to ensure even distribution across shards.. Uneven shard utilization with throttling on some shards is typically caused by a partition key that does not distribute records uniformly. Changing the partition key strategy—for example, by incorporating a random value or using a composite key—ensures records are spread evenly across all shards. This addresses the root cause without altering shard count or read capacity.

Answer

Use the MergeShards API to combine underutilized shards and reduce the total number of shards.

Answer

Increase the number of shards using the UpdateShardCount API to add more capacity.

Answer

Enable enhanced fan-out for the stream to provide dedicated throughput for consumers.

Question 11

A data engineer is using Amazon Kinesis Data Streams to ingest real-time data. The stream has 4 shards and is receiving 2 MB/s of data. The engineer notices that the WriteProvisionedThroughputExceeded metric is increasing. The engineer wants to resolve this issue with minimal changes. What should the engineer do?

Accepted Answer

Use a more random partition key to distribute data evenly across shards.. The WriteProvisionedThroughputExceeded metric indicates that some shards are exceeding their 1 MB/s write capacity. With 4 shards, total capacity is 4 MB/s, which is sufficient for 2 MB/s if evenly distributed. The issue is likely hot shards due to a non-uniform partition key. Using a more random partition key, such as a UUID, ensures even distribution and resolves the throttling without adding shards.

Answer

Increase the retention period of the stream to 168 hours.

Answer

Enable enhanced fan-out for the stream.

Answer

Increase the number of shards in the stream to 8.

Question 12

A data engineer maintains an AWS Glue Data Catalog table for an S3-based dataset. After new files were added, queries in Amazon Athena fail with the error 'HIVE_BAD_DATA: Error parsing field value for field 3: For input string: "N/A"'. The column is defined as bigint in the Data Catalog but contains the string 'N/A' in some records. Which action should the data engineer take to allow Athena to query the data without changing the underlying files?

Accepted Answer

Alter the table to set the column type to string in the AWS Glue Data Catalog.. The error is caused by a mismatch between the declared bigint type and actual string values in the files. The most direct fix that avoids rewriting data is to change the column type to string in the AWS Glue Data Catalog. Athena will then read the field as text, and the query will no longer attempt numeric conversion. Partition repair, projection, and SerDe tweaks do not address the type conversion failure.

Answer

Add a partition projection configuration for the table.

Answer

Run MSCK REPAIR TABLE on the table to refresh partitions.

Answer

Create a new table with the column as bigint and use a SerDe that skips malformed rows.

Question 13

A data engineer needs to schedule a daily AWS Glue job that extracts data from Amazon S3 and loads it into Amazon Redshift. The engineer wants to ensure the job runs at 2:00 AM UTC every day and can be monitored for failures. What is the simplest way to achieve this?

Accepted Answer

Use AWS Glue triggers to schedule the job with a cron expression.. AWS Glue triggers are the native scheduling mechanism for Glue jobs. Creating a scheduled trigger with a cron expression allows the job to run automatically at specified times. This approach is simple, requires no additional services, and provides monitoring through Glue job run statuses and CloudWatch metrics. The cron expression '0 2 * * ? *' schedules the job for 2:00 AM UTC daily.

Answer

Create an AWS Lambda function that triggers the Glue job using a cron expression in Amazon EventBridge.

Answer

Use Amazon EventBridge Scheduler to invoke the Glue job directly.

Answer

Create an AWS Step Functions state machine with a Wait state and a Glue job task.

Question 14

A data engineer manages an AWS Glue job that processes JSON files from Amazon S3 and writes to Amazon Redshift. The job fails with the error "Unable to find a suitable JDBC driver". The engineer has verified that the Glue connection to Redshift is configured correctly and the IAM role has the necessary permissions. What is the most likely cause of this error?

Accepted Answer

The Glue job's script is missing the required JDBC driver dependency, and the job's dependent jars path is not set.. AWS Glue jobs require the appropriate JDBC driver to connect to Amazon Redshift. The driver is not included in the Glue environment by default, so the engineer must provide it via the dependent jars path. The error indicates the driver class is missing, which is resolved by adding the driver JAR. Network and IAM issues would produce different errors.

Answer

The Redshift cluster is in a private subnet, and the Glue job cannot reach it due to a missing NAT gateway.

Answer

The Glue job's IAM role lacks permissions to access the Redshift cluster's metadata.

Answer

The Redshift cluster's security group does not allow inbound traffic from the Glue job's ENI.

Question 15

A data engineer is using AWS Step Functions to orchestrate a daily ETL workflow that includes an AWS Glue job, an Amazon EMR step, and an Amazon Redshift stored procedure. The workflow occasionally fails and the engineer needs to troubleshoot and recover. Which TWO actions should the engineer take to identify the failure and resume from the failed step? (Choose two.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Configure Retry and Catch fields on individual states.. Step Functions provides Retry and Catch fields at the state level to handle transient errors. By configuring Retry with a backoff strategy and maximum attempts, the workflow can automatically re-execute a failed state, such as a Glue job start or EMR step, without manual intervention. Catch can then define a fallback if retries are exhausted. This is the standard approach for making orchestration resilient to intermittent service issues.

Answer

Enable X-Ray tracing on the state machine.

Answer

Set the state machine's TimeoutSeconds to a large value.

Answer

Use a Map state to iterate over each Glue job.