DEA-C01 Data Operations and Support Practice Test 17 — 15 Questions

Question 1

A data engineer is using AWS Glue DataBrew to clean a dataset stored in Amazon S3. The engineer notices that the DataBrew job is failing with a permission error when trying to access the S3 bucket. The engineer has already verified that the IAM role associated with the DataBrew project has s3:GetObject and s3:ListBucket permissions on the bucket. Which additional factors could cause this permission error? (Choose two.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The S3 bucket is encrypted with an AWS KMS customer managed key, and the DataBrew role lacks kms:Decrypt permission.. Even with correct IAM role permissions, an S3 bucket policy with an explicit Deny can block access. Also, if the bucket is encrypted with SSE-KMS using a customer managed key, the DataBrew role needs kms:Decrypt permission. Both are common causes of access denied errors that are not resolved by S3 permissions alone. The other options are either already ruled out by the scenario or would produce different error types.

Answer

The DataBrew job is running in a VPC without an S3 VPC endpoint, and the bucket is not publicly accessible.

Answer

The S3 bucket is in a different AWS Region than the DataBrew project.

Answer

The DataBrew project is configured to use a different IAM role than the one the engineer checked.

Question 2

A data engineer is using Amazon Redshift and notices that a nightly ETL job is failing with 'Disk Full' errors. The cluster has 4 nodes of type dc2.large. The engineer needs to resolve the issue without changing the cluster size. Which action should the engineer take first?

Accepted Answer

Identify and drop unused tables or partitions, and run VACUUM to reclaim space.. The most direct and non-disruptive first step is to remove unnecessary data by dropping unused tables or partitions, which immediately frees disk space. Running VACUUM afterward reclaims space from deleted rows and re-sorts data. This combination resolves the disk full issue without changing the cluster size. Other options either violate the constraint or do not address the disk space problem.

Answer

Resize the cluster to add more nodes.

Answer

Run VACUUM to reclaim space from deleted rows.

Answer

Enable concurrency scaling to offload queries to additional clusters.

Question 3

A data engineer is using Amazon Athena to query data stored in Amazon S3. The engineer notices that some queries are slow and expensive due to full table scans. The data is in Parquet format and partitioned by date. The engineer wants to optimize query performance and reduce cost. Which two actions should the engineer take? (Choose two.)

Accepted Answer

Use the MSCK REPAIR TABLE command to update the table metadata with new partitions.. To optimize Athena queries on partitioned Parquet data, ensuring partition metadata is current via MSCK REPAIR TABLE and adding partitions on frequently filtered columns both enable partition pruning, which reduces data scanned and lowers cost. Other options like result reuse or format conversion do not address the root cause of full scans for new queries.

Answer

Use the AWS Glue Data Catalog to store table statistics for cost-based optimization.

Answer

Convert the data to ORC format to improve compression and query performance.

Answer

Enable Athena query result reuse to cache results of identical queries.

Question 4

A data engineer manages an Amazon Kinesis Data Streams application that uses AWS Lambda as a consumer. During a traffic spike, the Lambda function is throttled and the iterator age metric increases. The engineer must reduce iterator age without changing the producer. Which action should the engineer take?

Accepted Answer

Increase the Lambda function's reserved concurrency and batch size.. Iterator age reflects the lag between when a record is written and when the consumer processes it. Lambda throttling indicates the function cannot scale enough to keep up. Increasing reserved concurrency permits more parallel invocations, and a larger batch size processes more records per invocation, both reducing the backlog. These changes directly target the consumer bottleneck without modifying the producer or stream capacity.

Answer

Enable enhanced fan-out on the Kinesis data stream.

Answer

Increase the number of shards in the Kinesis data stream.

Answer

Switch the consumer to Kinesis Client Library (KCL) on Amazon EC2.

Question 5

A data engineer manages an AWS Glue PySpark job that reads JSON data from Amazon S3, transforms it, and writes Parquet to another S3 location. The job fails with the error 'AnalysisException: Unable to infer schema for JSON'. The S3 path contains a mix of well-formed JSON files and a few empty files (0 bytes). The engineer needs the job to process only valid files and ignore empty ones. What is the most operationally efficient solution?

Accepted Answer

Add a filter to the Glue job script to exclude empty files using the file size from the S3 object metadata before reading.. The most operationally efficient solution is to filter out empty files before the read operation. Empty files cause schema inference to fail because there is no data to infer from. By using S3 object metadata to exclude files with size 0, the job can proceed without modifying source data or adding unnecessary services. This approach is minimal, direct, and leverages existing AWS SDK capabilities within the Glue job script.

Answer

Set the Spark configuration 'spark.sql.files.ignoreMissingFiles' to true in the Glue job.

Answer

Enable the 'Ignore empty files' option in the AWS Glue job's advanced properties.

Answer

Convert the empty files to JSON with a null value using an AWS Lambda function triggered on S3 PUT events.

Question 6

A data engineer is configuring an AWS Glue crawler to catalog data stored in an Amazon S3 bucket. The data is partitioned by year, month, and day in the format s3://bucket/data/year=2023/month=01/day=01/. The crawler is taking a long time to complete and is scanning all partitions. The engineer wants to reduce crawler runtime and cost. Which configuration should the engineer use?

Accepted Answer

Set the crawler's 'RecrawlPolicy' to 'Crawl new folders only' and enable 'Update all new and existing partitions with metadata from the table'.. For a crawler scanning a large number of partitions, the most effective configuration is to set the recrawl policy to 'Crawl new folders only' so that only new partitions are processed on subsequent runs, and to enable 'Update all new and existing partitions with metadata from the table' so that existing partitions inherit metadata without being re-scanned. This reduces both runtime and cost.

Answer

Use partition indexes to speed up crawler execution.

Answer

Set the crawler's 'RecrawlPolicy' to 'Crawl new folders only'.

Answer

Configure the crawler to use a custom classifier that recognizes the partition structure.

Question 7

A data engineer is using AWS Step Functions to orchestrate an AWS Glue ETL job and an Amazon Redshift stored procedure. The state machine occasionally times out waiting for the Glue job to complete. The Glue job typically runs for 45 minutes. Which change should the engineer make to the state machine to prevent timeouts?

Accepted Answer

Replace the Glue job task with a synchronous .sync integration and set a TimeoutSeconds value.. Step Functions' optimized service integration for AWS Glue, invoked with the .sync suffix, runs the job and waits for completion. The task's TimeoutSeconds parameter controls how long the state waits before failing. Since the Glue job runs about 45 minutes, setting TimeoutSeconds above that value prevents the task from timing out prematurely, without needing custom polling or callbacks.

Answer

Use a Wait state with a fixed 45-minute delay before checking Glue job status.

Answer

Configure the Glue job to send a notification to Amazon SNS and use a callback task token.

Answer

Increase the Step Functions execution timeout to 24 hours.

Question 8

A data engineer needs to schedule a daily AWS Glue ETL job that processes data from Amazon S3 and writes to Amazon Redshift. The job must run every day at 2:00 AM UTC. Which AWS service should the engineer use to schedule the job?

Accepted Answer

AWS Glue triggers. AWS Glue triggers are the native scheduling mechanism for Glue jobs. They support cron expressions and can be configured to start jobs on a recurring basis. For a daily job at a specific time, a time-based trigger is the appropriate choice, as it directly integrates with the Glue job and handles job execution without additional services.

Answer

Amazon EventBridge (CloudWatch Events) rules

Answer

Amazon Managed Workflows for Apache Airflow (MWAA)

Answer

AWS Step Functions state machines

Question 9

A data engineer needs to schedule an AWS Glue job to run every day at 2:00 AM UTC. The job extracts data from Amazon S3, transforms it, and loads it into Amazon Redshift. The engineer wants a simple, serverless solution with minimal operational overhead. Which AWS service should the engineer use to schedule the job?

Accepted Answer

AWS Glue triggers. AWS Glue triggers provide a built-in, serverless scheduling mechanism specifically designed for Glue jobs. They support cron expressions and can be created with minimal configuration. This eliminates the need for additional services or infrastructure, aligning with the goal of minimal operational overhead.

Answer

Amazon EC2 with a cron job

Answer

Amazon EventBridge (CloudWatch Events) rules

Answer

AWS Step Functions with a Wait state

Question 10

A data engineer is using Amazon Kinesis Data Streams to ingest clickstream data. The stream has 10 shards and is receiving 2 MB/sec of data with an average record size of 1 KB. Consumers are AWS Lambda functions that process records and write to Amazon S3. The engineer observes that the Lambda function is being throttled and some records are processed multiple times. Which TWO actions should the engineer take to improve reliability and reduce duplicate processing? (Choose two.)

Accepted Answer

Set the Lambda event source mapping's ParallelizationFactor to a value greater than 1.. Lambda throttling and duplicate processing often stem from insufficient concurrency and low parallelism per shard. Increasing reserved concurrency allows more simultaneous invocations, while setting ParallelizationFactor above 1 lets multiple Lambda functions process a single shard concurrently. Together, these actions improve throughput and reduce retries that lead to duplicates. Shard increases and enhanced fan-out do not directly address Lambda-specific throttling.

Answer

Configure the Lambda event source mapping to start with the TRIM_HORIZON position and enable checkpointing.

Answer

Enable enhanced fan-out on the Kinesis stream.

Answer

Increase the number of shards in the Kinesis stream.

Question 11

A data engineer is orchestrating an AWS Glue ETL job that must run only after an upstream Amazon Redshift stored procedure finishes loading a staging table. The Glue job currently starts on a schedule and sometimes reads incomplete data. The engineer needs to trigger the Glue job automatically when the Redshift stored procedure completes. Which solution meets these requirements with the LEAST operational overhead?

Accepted Answer

Configure an Amazon EventBridge rule that matches the Redshift Data API event pattern for the stored procedure completion, and set the Glue job as the rule target.. The Redshift Data API publishes statement-completion events to Amazon EventBridge, so an EventBridge rule can invoke the Glue job as soon as the stored procedure succeeds. This removes polling, scheduled guessing, and manual intervention, giving the lowest operational overhead and correct ordering. The other approaches either poll, guess timing, or rely on humans, all of which fail to guarantee the job starts exactly after the load completes.

Answer

Create an AWS Lambda function on a 5-minute schedule that queries the Redshift system tables to check whether the stored procedure has finished, then calls StartJobRun on the Glue job.

Answer

Enable Amazon Redshift event notifications to an Amazon SNS topic, and subscribe an email endpoint that a data engineer monitors manually to start the Glue job.

Answer

Use an AWS Glue trigger of type SCHEDULED with a cron expression set a few minutes after the expected Redshift procedure completion time.

Question 12

A data engineer maintains a fleet of AWS Glue Python shell jobs that extract data from an Amazon RDS for PostgreSQL database and write Parquet files to Amazon S3. Job runs started failing intermittently with the message 'Unable to execute HTTP request: Connect timed out' when connecting to RDS. CloudWatch metrics show that RDS CPU utilization is normal, but the Glue job's network traffic spikes during the extraction phase. The Glue jobs run in a development endpoint subnet with a NAT gateway. Which action will most likely resolve the connection timeouts while preserving the Glue job's serverless execution model?

Accepted Answer

Attach the Glue jobs to a VPC subnet with a route to the RDS instance via VPC peering or a private link, and adjust the RDS security group to allow the Glue job's security group.. The timeout occurs because the Glue job's network path to RDS traverses a NAT gateway, which can introduce latency and connection failures. Moving the Glue job into a subnet with a private route to RDS, such as VPC peering or AWS PrivateLink, and ensuring the RDS security group allows the Glue job's security group, establishes a reliable private connection without changing the serverless nature of Glue.

Answer

Increase the number of Glue DPUs allocated to each job run.

Answer

Increase the RDS instance class to a larger size with more vCPUs.

Answer

Enable Glue job bookmarks to track previously processed data.

Question 13

A data engineer is using Amazon EventBridge to trigger an AWS Lambda function when an object is created in an Amazon S3 bucket. The engineer wants to ensure that the Lambda function is invoked only for objects with the prefix 'processed/' and suffix '.csv'. Which EventBridge event pattern should the engineer use?

Accepted Answer

A pattern that matches the S3 bucket name and the object key using prefix and suffix conditions.. EventBridge event patterns allow filtering on the S3 object key using prefix and suffix fields. By specifying the bucket name and the key conditions, the rule will only match objects with the 'processed/' prefix and '.csv' suffix. This ensures the Lambda function is invoked only for the relevant files. Wildcards, event name, and object size do not provide the required filtering.

Answer

A pattern that matches the S3 bucket name and the object size.

Answer

A pattern that matches the S3 bucket name and uses a wildcard in the object key.

Answer

A pattern that matches the S3 bucket name and the event name 'PutObject'.

Question 14

A data engineer manages an AWS Step Functions state machine that orchestrates a nightly pipeline of AWS Glue jobs and Amazon EMR steps. Occasionally a Glue job returns a non-zero exit condition that should halt the pipeline, but the state machine continues to the next state and downstream tables are left partially loaded. The engineer needs the workflow to stop and record the failure. Which change addresses this?

Accepted Answer

Configure the Glue task state with a Catch field that transitions to a Fail state on the error name returned by the job.. Step Functions task states succeed or fail based on the response from the integrated service. Without a Catch field, an error surfaced by a Glue job can be handled by default retry behavior and then either fail the state or, if the error is not surfaced as a task failure, allow the next state to run. Adding a Catch that routes to a Fail state guarantees the workflow halts and records the cause.

Answer

Set the Glue job's MaximumConcurrency to 1 so overlapping runs cannot interfere with each other.

Answer

Enable CloudWatch Logs on the state machine and create an alarm that emails the operations team on job errors.

Answer

Add a Retry field with an exponential backoff to the Glue task state so transient errors are absorbed.

Question 15

A data engineer is using Amazon Kinesis Data Streams to ingest clickstream data. The stream has 4 shards and is receiving 2 MB/sec of data with 2000 records/sec. Consumers are reading with the Kinesis Client Library (KCL) and processing records. The engineer observes that the consumer lag is increasing, and some shards are not being processed at all. What is the MOST likely cause?

Accepted Answer

The KCL worker's maxLeasesForWorker or maxLeasesToStealAtOneTime configuration is too low, preventing it from acquiring leases for all shards.. The Kinesis Client Library assigns shards to workers via leases. If maxLeasesForWorker is set too low, a worker cannot take leases for all shards it could handle, leaving some shards without a consumer. Adjusting this configuration ensures all shards are processed, which resolves the lag and unprocessed shards.

Answer

The KCL application is not using a unique application name, causing multiple consumers to compete for the same shards and some to be starved.

Answer

The number of KCL worker instances is less than the number of shards, so some shards are not assigned to any worker.

Answer

The Kinesis stream's shard count is insufficient for the incoming data rate, causing throttling and uneven processing.