DEA-C01 Data Operations and Support Practice Test 14 — 15 Questions

Question 1

Your company has an Amazon S3-based data lake partitioned by year/month/day. An AWS Glue crawler runs daily to update the Data Catalog. A Spark job on Amazon EMR reads the latest partition and performs transformations. Recently, the Spark job has been failing with a 'FileNotFoundException' for a file that is expected to exist. You check the S3 bucket and see that the file exists. The job is configured to use S3 as the direct input source with EMRFS consistent view enabled. The IAM role for the EMR cluster has full S3 access. What is the most likely cause?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The EMR cluster is experiencing eventual consistency issues despite EMRFS consistent view.. The correct answer is C: the EMR cluster is experiencing eventual consistency issues despite EMRFS consistent view. EMRFS consistent view (with DynamoDB-backed metadata) only tracks objects written through EMRFS itself; files written by external writers such as the Glue crawler or other tools are not tracked, so a Spark job can see stale metadata and throw FileNotFoundException even though the object exists in S3. Option A is wrong because a crawler running after the job would affect catalog freshness, not the direct S3 read path used here. Option B is wrong because the IAM role has full S3 access, so crawler permissions are not the issue. Option D is wrong because the file is confirmed to still exist in the bucket, ruling out lifecycle deletion.

Answer

The Data Catalog partition metadata is out of sync because the crawler runs after the job.

Answer

The Glue crawler did not update the Data Catalog because of insufficient permissions.

Answer

The file was deleted by a lifecycle policy after the job started.

Question 2

Your organization uses Amazon Redshift for analytical workloads. You have noticed that queries are slow on a large fact table. The table is distributed by KEY on the customer_id column and sorted by transaction_date. The table is frequently updated with new records. To improve query performance, you decide to implement a distribution style that reduces data movement. Which action should you take?

Accepted Answer

Change the distribution style to EVEN to distribute rows evenly across all nodes.. The table is distributed by KEY on customer_id, which means rows with the same customer_id are collocated on the same node. However, if the table is frequently updated and queries are slow, the issue is likely that the distribution key causes data skew or that the join patterns don't align with the distribution key. Changing to EVEN distribution spreads rows evenly across all nodes, which reduces data movement for queries that don't filter or join on customer_id, and eliminates skew. EVEN is the best choice when the table is not typically joined on the distribution key or when the distribution key has low cardinality or skew. Since the table is a large fact table with frequent updates, EVEN ensures balanced storage and processing across nodes, improving overall query performance by minimizing data movement during parallel operations.

Answer

Change the distribution style to ALL to put a copy of the table on every node.

Answer

Change the distribution style to AUTO to let Redshift choose the best distribution.

Answer

Change the sort key to include customer_id as well.

Question 3

Your company runs a data pipeline that ingests data from AWS Database Migration Service (DMS) into Amazon S3 in Parquet format. An AWS Glue ETL job then transforms the data and loads it into an Amazon Redshift cluster. The Glue job uses a JDBC connection to Redshift. Recently, the Glue job started failing with a 'communication failure' error when writing to Redshift. The Redshift cluster is in a VPC with public accessibility disabled. The Glue job runs in a VPC with a subnet that has a route to a NAT gateway. The security group for Redshift allows inbound traffic from the Glue job's security group. The Glue job's IAM role has the necessary permissions. What is the most likely cause?

Accepted Answer

The Glue job and Redshift cluster are in different VPCs that are not peered or connected via VPC Transit Gateway.. Even though the security group allows inbound traffic, the Glue job's VPC may not have a route to the Redshift cluster's VPC if they are in different VPCs. Option A is wrong because IAM permissions are not the issue. Option B is wrong because the Redshift cluster is in a VPC and not publicly accessible. Option D is wrong because the NAT gateway is for outbound internet, not for connecting to Redshift within the same VPC.

Answer

The Glue job's IAM role does not have the redshift:DescribeClusters permission.

Answer

The Redshift cluster's public accessibility is disabled, but the Glue job is trying to connect over the internet.

Answer

The NAT gateway is not configured to allow traffic to the Redshift cluster's subnet.

Question 4

A data engineering team uses AWS Glue ETL jobs to process daily data from an Amazon RDS for PostgreSQL instance into Amazon S3. Recently, the jobs have been failing randomly with the error 'psycopg2.OperationalError: could not connect to server: Connection timed out'. The RDS instance is in a private subnet with a security group that allows inbound traffic from the Glue job's security group on port 5432. The Glue job is configured to use the same VPC, subnet, and security group. The RDS instance has sufficient connections and is not at CPU or memory limits. The failures occur at different times each day, and the job works when retried immediately. Which action should the team take to resolve the issue?

Accepted Answer

Increase the number of retries in the Glue job configuration to 5.. The intermittent 'Connection timed out' errors that resolve on immediate retry are characteristic of transient network issues or brief RDS connection spikes, not a permanent misconfiguration. AWS Glue jobs have a default retry count of 0, so any transient failure causes the job to fail permanently. By increasing the number of retries to 5, the job will automatically reattempt the connection, allowing it to succeed when the transient condition clears. This is the most direct and appropriate fix for random, non-persistent failures.

Answer

Set the Glue job timeout to 60 minutes to ensure the job does not fail prematurely.

Answer

Change the subnet of the Glue job to one with a larger CIDR range (e.g., /20 instead of /24).

Answer

Add an inbound rule to the RDS security group allowing traffic on ports 1024-65535 from the Glue security group.

Question 5

A company uses Amazon Kinesis Data Streams to ingest clickstream data from web applications. The data is consumed by a Kinesis Data Analytics application that runs SQL queries. Recently, the data analytics application has been falling behind, and the 'MillisBehindLatest' metric for the consumer has been increasing steadily. The shard count is 4, and the average records per second per shard is 200, with an average record size of 1 KB. The provisioned shard limit for the account is 10. Which action will resolve the issue?

Accepted Answer

Increase the number of shards in the Kinesis stream to 8.. The issue is that the Kinesis Data Analytics application is falling behind because the stream does not have enough shards to handle the incoming data rate. With 4 shards, each shard can handle up to 1 MB/s or 1000 records/s. The current load is 200 records/s per shard with 1 KB records, which is 200 KB/s per shard, well within limits. However, the analytics application might be limited by the number of shards for parallel processing. Increasing shards to 8 allows more parallelism and throughput. Also, the provisioned shard limit is 10, so 8 is feasible.

Answer

Enable enhanced fan-out on the Kinesis stream and subscribe the analytics application to it.

Answer

Reduce the checkpoint interval on the Kinesis Client Library (KCL) consumer to commit offsets more frequently.

Answer

Increase the provisioned write capacity of the Kinesis stream by requesting a shard limit increase.

Question 6

A data engineer is troubleshooting a failed AWS Glue ETL job that reads from Amazon S3 and writes to Amazon Redshift. The job fails with the error: 'ERROR: Cannot insert a duplicate key into unique index'. The Redshift table has a primary key on the 'id' column. The data in S3 contains multiple records with the same 'id'. The engineer needs to ensure that only the latest record for each 'id' is loaded into Redshift. The data has a 'timestamp' column. Which approach should the engineer take?

Accepted Answer

Use the 'dropDuplicates' transformation in the Glue ETL script, ordering by 'timestamp' descending to keep the latest record for each 'id'.. Using the AWS Glue 'dropDuplicates' transformation on the 'id' column, ordering by 'timestamp' descending, will remove duplicate 'id' values, keeping the latest record. This is the direct approach within the Glue ETL script. Option B is incorrect because using 'overwrite' mode would replace the entire Redshift table, which would discard existing data and is not a targeted solution for handling duplicates. Option C is incorrect because loading into a staging table and using MERGE requires additional setup and is not directly part of the Glue ETL script, making it less efficient for the stated requirement. Option D is incorrect because disabling primary key constraints only defers the duplicate key error, and it would compromise data integrity rather than properly deduplicating records.

Answer

Set the write mode to 'overwrite' in the Glue job to replace the entire Redshift table.

Answer

Load data into a staging table in Redshift and then use a MERGE operation to insert only new records.

Answer

Disable primary key constraints on the Redshift table before loading.

Question 7

A company runs a data pipeline using AWS Step Functions to orchestrate multiple AWS Lambda functions and AWS Glue jobs. The pipeline processes large CSV files from Amazon S3, transforms them, and loads them into Amazon Redshift. Recently, the pipeline has been failing intermittently with a 'StateMachineExecutionLimitExceeded' error. The error occurs when multiple pipeline runs are triggered simultaneously. The current execution limit for the state machine is 1000. The team expects up to 200 concurrent executions during peak hours. Which action should the team take to resolve the issue?

Accepted Answer

Request a service quota increase for the maximum number of state machine executions from AWS Support.. The error 'StateMachineExecutionLimitExceeded' indicates that the number of concurrent executions for the state machine has hit the service quota, which is 1000 by default. Since the team expects up to 200 concurrent executions, they are well within the default limit, but the error suggests the limit might have been lowered or they are hitting a different limit. However, the correct action is to request a quota increase because the error explicitly states the limit is exceeded, and the default limit is 1000, so if they are hitting it, they need more. The other options do not address the root cause of the error.

Answer

Increase the execution timeout for the state machine to 1 hour.

Answer

Increase the Lambda function concurrency limits to allow more parallel processing.

Answer

Implement a queue (e.g., Amazon SQS) to buffer the pipeline triggers and process them sequentially.

Question 8

A company uses Amazon S3 to store log files from multiple sources. The logs are partitioned by year, month, day, and hour. A data engineer uses Amazon Athena to query the logs. Recently, users have reported that queries are taking longer than expected. The engineer notices that many queries are scanning large amounts of data even when filtering on partition columns. The total data size is 10 TB, and the average query scans 2 TB. The partition columns are properly defined in the table schema. What is the most likely cause of the slow queries?

Accepted Answer

The table is not partitioned, or the partitions are not properly defined in the table DDL.. If queries filter on partition columns but still scan 2 TB out of 10 TB, the partitions are likely not registered in the table metadata (e.g., missing MSCK REPAIR TABLE or partition projection), so Athena falls back to scanning the full table. Properly defined partitions in the DDL would let Athena prune to only the relevant S3 prefixes.

Answer

The number of partitions is too large, causing Athena to spend time listing partitions.

Answer

The log files are stored in compressed format (e.g., gzip), which increases the amount of data scanned.

Answer

The log files are stored in CSV format instead of columnar formats like Parquet.

Question 9

A data engineer is designing a data lake on Amazon S3 that will be used for both batch processing with Amazon EMR and interactive queries with Amazon Athena. The data includes sensitive personally identifiable information (PII) that must be encrypted at rest. The company requires that the encryption keys be managed by the company and rotated every 90 days. Which TWO options should the engineer implement to meet these requirements? (Choose TWO.)

Accepted Answer

Enable S3 default encryption using SSE-KMS with the customer managed key.. The correct answers are C and D. Option C enables S3 default encryption using SSE-KMS with a customer managed key, ensuring all objects are encrypted at rest with keys managed by the company. Option D creates a customer managed key in AWS KMS with automatic yearly rotation, which satisfies the requirement for key rotation; the company can also perform manual rotations every 90 days if needed. Option A is incorrect because SSE-C requires the customer to manage the keys themselves, including storing them in Secrets Manager, and does not provide automatic rotation. Option B is incorrect because a bucket policy can enforce encryption but does not manage keys. Option E is incorrect because SSE-S3 uses Amazon-managed keys, not customer-managed keys.

Answer

Use customer-provided keys (SSE-C) and store the keys in AWS Secrets Manager.

Answer

Configure a bucket policy to deny uploads that are not encrypted.

Answer

Use S3 managed keys (SSE-S3) for server-side encryption.

Question 10

A company runs an Amazon Redshift cluster for data warehousing. The data engineering team notices that the 'Amazon Redshift Data API' is timing out when executing long-running queries. The queries typically take more than 10 minutes to complete. The team wants to ensure that the queries can complete without timeout and that the results are retrievable. Which TWO steps should the team take? (Choose TWO.)

Accepted Answer

Use the 'GetStatementResult' operation to retrieve results after the query completes.. Option C is correct because the Amazon Redshift Data API executes queries asynchronously; after a long-running query finishes, the team must call GetStatementResult with the statement ID to fetch the result set, since results are not returned inline for asynchronous executions. Option E is correct because the Data API supports asynchronous execution: passing a StatementName (or simply not waiting synchronously) lets the query run in the background, and the team can poll DescribeStatement until the status is FINISHED, avoiding the client-side timeout that occurs when waiting on a synchronous call. Option A is incorrect because the Data API has no QueryExecutionTimeout parameter; timeouts are governed by the client/SDK and the service's asynchronous model, not a configurable call parameter. Option B is incorrect because there is no 'timeout' parameter in Redshift cluster configuration that extends Data API query execution. Option D is incorrect because max_execution_time is a Redshift workload-management/query monitoring parameter that aborts queries exceeding the limit, which would terminate rather than allow the 10+ minute queries to complete.

Answer

Set the 'QueryExecutionTimeout' parameter in the Data API call to 30 minutes.

Answer

Increase the 'timeout' parameter in the Redshift cluster configuration.

Answer

Set the 'max_execution_time' parameter in the Redshift parameter group to 30 minutes.

Question 11

A data engineer is monitoring an AWS Glue ETL job that processes data from Amazon DynamoDB to Amazon S3. The job is taking longer than expected. The engineer suspects that the job's parallelism is not optimal. Which THREE actions can improve the job's performance? (Choose THREE.)

Accepted Answer

Enable the 'groupFiles' option in the S3 sink to coalesce small files.. Enabling 'groupFiles' in the S3 sink coalesces small files into larger ones, reducing the number of write operations and improving write performance. Option C is correct because increasing MaxCapacity (DPU) allocates more processing units, increasing parallelism and processing speed. Option E is correct because increasing 'dynamodb.throughput.read.percentage' allocates a higher percentage of the table's provisioned read capacity to the Glue job, allowing more parallel reads from DynamoDB. Option B is incorrect because decreasing 'dynamodb.splits' reduces the number of parallel readers, which can lower parallelism and slow down the job. Option D is incorrect because disabling job bookmarks results in reprocessing all data each run, which increases processing time and does not improve performance.

Answer

Decrease the 'dynamodb.splits' parameter to reduce the number of parallel readers.

Answer

Disable job bookmark to avoid storing metadata.

Question 12

A data engineer is using AWS Step Functions to orchestrate a series of AWS Glue jobs. The engineer notices that some Glue jobs fail intermittently due to transient network issues when accessing Amazon S3. The engineer wants to implement a retry mechanism at the Step Functions level. Which two configurations in the Step Functions state machine will achieve this? (Choose two.)

Accepted Answer

Add a Retry field to the state that invokes the Glue job, specifying the error name 'States.TaskFailed' and a maximum attempts of 3.. Step Functions natively supports retries via the Retry field. To handle transient Glue job failures, you can specify either a specific Glue error like 'Glue.AWSGlueException' or the general 'States.TaskFailed'. Both will trigger retries with configurable intervals and backoff. This keeps the retry logic within the state machine, providing visibility and control.

Answer

Use a Map state to run the Glue job multiple times in parallel and take the first successful result.

Answer

Configure the Glue job to have a higher number of retries in its job definition, and Step Functions will automatically inherit that retry policy.

Answer

Add a Catch field to the state that transitions to a fallback state that restarts the entire Step Functions execution.

Question 13

A data engineer is using AWS Step Functions to orchestrate a data pipeline that includes an AWS Glue job, an Amazon EMR step, and an Amazon Redshift stored procedure. The pipeline occasionally fails at the Redshift step with a timeout error. The engineer wants to implement a retry strategy that waits for the Redshift cluster to be available before retrying. Which Step Functions state configuration should the engineer use?

Accepted Answer

Use a Task state with a Retry field that includes a MaxDelaySeconds and a BackoffRate, and catch the timeout error to loop back to a state that checks cluster status.. The most reliable way to handle a timeout due to an unavailable Redshift cluster is to implement a retry with exponential backoff and a status check loop. This ensures the workflow waits dynamically for the cluster to be ready, retrying only when necessary, and avoids unnecessary delays or duplicate work.

Answer

Use a Retry field with a BackoffRate and MaxAttempts, and set the IntervalSeconds to a fixed value.

Answer

Use a Wait state with a fixed time before the Redshift step, and then retry the entire pipeline.

Answer

Use a Parallel state to run the Redshift step concurrently with a Lambda function that waits for the cluster.

Question 14

A data engineer is monitoring an AWS Glue ETL job that reads from Amazon S3 and writes to Amazon Redshift. The job has been running successfully but recently started taking longer to complete. The engineer checks the job metrics and notices increased 'glue.driver.aggregate.bytesRead' and 'glue.driver.aggregate.elapsedTime'. The engineer suspects that the input data has grown significantly. Which TWO actions should the engineer take to improve the job's performance? (Choose two.)

Accepted Answer

Increase the number of workers in the Glue job configuration.. The job is experiencing increased data volume, leading to longer runtimes. To improve performance, the engineer should scale the job horizontally by increasing the number of workers, and optimize data reading by partitioning the input data in S3 and using partition columns. These actions directly address the increased data volume by parallelizing processing and reducing data scanned.

Answer

Increase the timeout value of the Glue job.

Answer

Switch the Glue job to use a smaller worker type to reduce cost.

Answer

Enable job bookmarks to track previously processed data.

Question 15

A data engineer is using AWS Lake Formation to manage access to a data lake in Amazon S3. The engineer grants SELECT permission on a table to an IAM user, but the user still cannot query the table using Amazon Athena. The engineer verifies that the user has the necessary IAM permissions for Athena and S3. What is the most likely cause of the issue?

Accepted Answer

The user needs to be granted DESCRIBE permission on the database in Lake Formation.. In Lake Formation, permissions are hierarchical. To query a table, a principal needs SELECT on the table and DESCRIBE on the database. The absence of DESCRIBE on the database prevents the user from seeing the table in Athena, leading to query failures. The correct action is to grant DESCRIBE on the database.

Answer

The user must have the 'lakeformation:GetDataAccess' permission in their IAM policy.

Answer

The user needs to be granted ALL permissions on the table in Lake Formation.

Answer

The S3 bucket must be registered with Lake Formation, and the user must have IAM permissions to the bucket.