DEA-C01 Data Operations and Support Practice Test 19 — 15 Questions

Question 1

A data engineer is designing a data pipeline using AWS Step Functions to orchestrate multiple AWS Glue ETL jobs. The pipeline must handle failures and retries. Which TWO configurations should the engineer use to ensure the pipeline is resilient? (Choose two.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Configure the state machine to use a 'Catch' rule to handle specific errors and transition to a fallback state. To ensure resilience in AWS Step Functions, the engineer should configure a Catch rule to handle errors by transitioning to a fallback state (Option B) and define a Timeout for each state to prevent the pipeline from hanging indefinitely (Option D). A dead-letter queue (DLQ) is not directly used by Step Functions but by services like Lambda. A fixed retry interval is less effective than exponential backoff. Using a Parallel state is for concurrency, not resilience.

Answer

Configure a dead-letter queue (DLQ) for the state machine

Answer

Set the 'Retry' interval to a fixed value instead of exponential backoff

Answer

Use a 'Parallel' state to run multiple Glue jobs simultaneously

Question 2

A company stores sensitive data in Amazon S3 and requires that all data be encrypted at rest. The data is accessed by multiple AWS services. Which solution meets the encryption requirement with the LEAST operational overhead?

Accepted Answer

Enable S3 default encryption with SSE-S3. Enabling S3 default encryption with SSE-S3 provides server-side encryption automatically without any key management overhead. Option A is wrong because SSE-KMS requires managing KMS keys. Option B is wrong because client-side encryption requires managing keys on the client side. Option C is wrong because SSE-C requires managing your own keys.

Answer

Use server-side encryption with AWS KMS (SSE-KMS)

Answer

Use client-side encryption with AWS KMS

Answer

Use server-side encryption with customer-provided keys (SSE-C)

Question 3

A company uses Amazon Kinesis Data Analytics for Apache Flink to process streaming data. The application is experiencing high latency and checkpoint failures. Which THREE actions should the data engineer take to improve performance and reliability? (Choose three.)

Accepted Answer

Increase the parallelism of the Flink application. Options A, B, and C are correct. Option A: Increasing parallelism improves throughput by distributing workload across more resources. Option B: Using event time processing helps handle out-of-order data and can reduce latency and checkpoint failures by allowing more accurate watermarks. Option C: Increasing the checkpoint interval reduces the frequency of checkpoint operations, which can reduce checkpoint failures under high load. Option D is incorrect because decreasing parallelism reduces throughput, worsening latency. Option E is incorrect because disabling checkpointing removes fault tolerance and does not improve performance.

Answer

Decrease the parallelism to reduce resource contention

Answer

Disable checkpointing to avoid checkpoint failures

Question 4

A data pipeline using Amazon Kinesis Data Streams is experiencing high consumer lag. The stream has 10 shards. The consumer is an AWS Lambda function that processes each record and writes to Amazon DynamoDB. What is the MOST likely cause of the lag?

Accepted Answer

The Lambda function's reserved concurrency is set too low. The most likely cause of high consumer lag is that the Lambda function's reserved concurrency is set too low (Option A). Each Kinesis shard is processed by a single Lambda invocation, and if the function's concurrency limit is less than the number of shards (10), some shards will not be processed in parallel, leading to lag. Option B (DynamoDB write capacity throttling) could cause lag but is less common if the table is properly provisioned. Option C (insufficient shards) is unlikely because 10 shards already provide parallelism; increasing shards would improve throughput only if Lambda concurrency is not the bottleneck. Option D (authorization) would cause errors, not just lag.

Answer

The DynamoDB table's write capacity is throttling writes

Answer

The number of shards is insufficient for the data volume

Answer

The Lambda function is not authorized to read from Kinesis

Question 5

A data engineer is setting up Amazon CloudWatch alarms for an Amazon Redshift cluster. The engineer wants to be alerted when the disk space usage exceeds 80% for more than 5 minutes and when the CPU utilization exceeds 90% for more than 10 minutes. Which TWO CloudWatch metrics and conditions should the engineer use? (Choose two.)

Accepted Answer

Metric: CPUUtilization; Condition: > 90 for 10 minutes. The correct metrics and conditions are: for disk space usage exceeding 80% for more than 5 minutes, use PercentageDiskSpace with a threshold of 80 and a period of 5 minutes; for CPU utilization exceeding 90% for more than 10 minutes, use CPUUtilization with a threshold of 90 and a period of 10 minutes. These correspond to options A and E. Option B (DatabaseConnections) monitors connections, not disk or CPU. Option C (NetworkReceiveThroughput) measures network traffic. Option D (WLMQueueLength) tracks query queue length, not disk or CPU.

Answer

Metric: DatabaseConnections; Condition: > 500 for 5 minutes

Answer

Metric: NetworkReceiveThroughput; Condition: > 1 GB for 10 minutes

Answer

Metric: WLMQueueLength; Condition: > 100 for 5 minutes

Question 6

A data engineer needs to transform a large dataset stored in Amazon S3 using Apache Spark. The engineer wants to minimize startup time and use a serverless approach. Which AWS service should the engineer use?

Accepted Answer

AWS Glue. AWS Glue provides a serverless Spark environment with fast startup. Option A is wrong because Amazon Redshift is a data warehouse, not a Spark environment. Option B is wrong because Amazon EMR requires cluster provisioning, which increases startup time. Option D is wrong because Amazon Athena is for querying data, not for transforming with Spark.

Answer

Amazon Redshift

Answer

Amazon EMR

Answer

Amazon Athena

Question 7

A data engineer is responsible for a data pipeline that uses Amazon S3 as a data lake, AWS Glue for ETL, and Amazon Athena for ad-hoc queries. The pipeline ingests CSV files from an external partner via SFTP into an S3 bucket. The files are then processed by a Glue job that converts them to Parquet and writes to a separate S3 bucket partitioned by date. The Glue job runs daily and is triggered by a scheduled CloudWatch Events rule. Recently, the data engineer noticed that some days the Glue job fails because of memory errors, and on those days the Athena queries that rely on the data return incomplete results. The engineer needs to ensure that the pipeline is resilient and that Athena queries always see a complete view of the data, even if the Glue job fails mid-run. The engineer also needs to minimize re-processing of data. Which course of action should the engineer take?

Accepted Answer

Modify the Glue job to use job bookmarks for incremental processing and write the Parquet output to a temporary location, then use an S3 copy operation to move the data into the final partitioned location only after the job completes successfully.. Using Glue job bookmarks enables incremental processing and the ability to resume from the last successful checkpoint. Staging the data in a temporary location and moving it atomically ensures that Athena sees only complete data, even if the job fails mid-run. Option A is wrong because increasing worker capacity does not prevent partial writes during failures. Option B is wrong because using Lambda for conversion is less scalable and error-prone, and it still doesn't solve the atomicity issue. Option D is wrong because partition projection does not address the atomicity of writes after job failures.

Answer

Increase the number of workers and the worker type to G.2X to handle the memory errors, and enable job retries.

Answer

Replace the Glue job with an AWS Lambda function that processes the CSV files and writes Parquet to S3, and use S3 Event Notifications to trigger the function.

Answer

Use Athena partition projection to automatically discover partitions and set up a retry mechanism using AWS Step Functions.

Question 8

A company uses AWS DMS to migrate data from an on-premises Oracle database to Amazon Aurora MySQL. After the migration, the data in Aurora is inconsistent with the source. The engineer needs to ensure ongoing replication with minimal downtime. Which solution should the engineer implement?

Accepted Answer

Configure a DMS task with change data capture (CDC). Configuring a DMS task with Change Data Capture (CDC) enables ongoing replication of changes from the source Oracle database to the target Aurora MySQL database with minimal downtime, ensuring consistency. Option A is incorrect because AWS Schema Conversion Tool (SCT) only converts schema and does not handle data replication. Option B is incorrect because mysqldump provides a one-time export/import, not ongoing replication. Option D is incorrect because performing a full load migration again would disrupt operations and would not capture ongoing changes.

Answer

Use AWS Schema Conversion Tool (SCT) to convert the schema

Answer

Export the data from Oracle and import into Aurora using mysqldump

Answer

Perform a full load migration again

Question 9

A data engineering team notices that an AWS Glue ETL job, which processes hourly data from an S3 bucket, is taking progressively longer to run. The job reads Parquet files partitioned by date and hour. Which action is MOST likely to improve the job's performance?

Accepted Answer

Enable pushdown predicate filtering on the job's data source.. Enabling pushdown predicate filtering allows the Glue job to read only the relevant partitions (e.g., specific date and hour) instead of scanning all data. This directly addresses the symptom of progressively longer run times as data accumulates. Option B is incorrect because Parquet is a columnar format optimized for performance, while CSV would increase I/O. Option C, increasing DPUs, can improve parallelism but does not reduce the amount of data read, so it may not address the root cause. Option D, switching to Python shell, would lose Spark's distributed processing capabilities and is unlikely to improve performance.

Answer

Convert Parquet files to CSV to improve read performance.

Answer

Increase the number of DPUs for the job.

Answer

Switch from Spark to Python shell for simpler processing.

Question 10

A company runs a time-series forecasting model that writes results to an S3 bucket every 5 minutes. A downstream ETL job reads this data, but sometimes fails because it encounters incomplete files (zero bytes). What is the MOST reliable way to ensure the ETL job only processes complete files?

Accepted Answer

Use S3 Event Notifications to trigger a Lambda function that checks file size and then moves the file to a 'ready' prefix.. S3 Event Notifications can trigger a Lambda function upon object creation, which can check the file size (e.g., > zero bytes) and then copy the file to a 'ready' prefix, ensuring the ETL job only processes complete files. Option A is wrong because an S3 Lifecycle policy can delete small files but does not prevent the ETL from reading incomplete files. Option B is wrong because S3 Copy does not verify completeness. Option C is wrong because S3 Select still reads the file even if it is incomplete; it doesn't guarantee completeness.

Answer

Set an S3 Lifecycle policy to delete files smaller than 1 MB.

Answer

Use S3 Copy to move files to a 'processed' folder after the ETL job reads them.

Answer

Configure S3 Select to query the files and only return rows if the file is complete.

Question 11

A data engineer needs to troubleshoot why an AWS Glue job is failing with a 'Insufficient Memory' error. The job processes a 10 GB dataset. Which step should the engineer take FIRST?

Accepted Answer

Increase the number of DPUs allocated to the job.. The correct first step is to increase the number of DPUs allocated to the job. The 'Insufficient Memory' error indicates that the current memory per worker is insufficient for processing the 10 GB dataset. Increasing DPUs provides more memory and compute resources, directly addressing the issue. Option A is incorrect because Python shell jobs are single-threaded and cannot handle large datasets efficiently; Glue Spark jobs are designed for distributed processing. Option B (repartitioning) can improve performance but does not add memory; it may help after resource allocation is adequate. Option C is invalid because AWS Glue does not support Java jobs; it supports Python and Scala.

Answer

Switch from using Apache Spark to Python shell.

Answer

Repartition the data into more partitions within the job.

Answer

Change the job type from Python to Java.

Question 12

A data engineer needs to schedule a daily ETL job that runs on Amazon EMR. The job should be triggered automatically and send an email on failure. Which AWS service should the engineer use to orchestrate the job?

Accepted Answer

AWS Step Functions. WS Step Functions (Option B). Step Functions is a serverless workflow service that can orchestrate multiple AWS services, including EMR steps, into a state machine. It can be triggered on a schedule using Amazon EventBridge or CloudWatch Events, and it can integrate with Amazon SNS to send email notifications on failure. Option A (Amazon EventBridge) is an event bus that can trigger services but does not provide built-in orchestration for complex workflows. Option C (SQS) is a message queue service and not an orchestrator. Option D (Amazon CloudWatch Events) is similar to EventBridge and can trigger Lambda functions but lacks native workflow orchestration capabilities.

Answer

Amazon EventBridge

Answer

Amazon Simple Queue Service (SQS)

Answer

Amazon CloudWatch Events

Question 13

A company is using Amazon Redshift for its data warehouse. A data engineer notices that COPY commands from S3 are failing intermittently with 'S3ServiceException: Access Denied'. The IAM role used by Redshift has the correct permissions. What is the MOST likely cause?

Accepted Answer

The S3 bucket policy denies access to the Redshift cluster's IP addresses.. S3 bucket policies may deny access even if the role allows it. Option A is wrong because the role is already attached. Option B is wrong because encryption would cause different errors. Option C is wrong because if the role exists, it should work; the issue is likely external.

Answer

The IAM role is not attached to the Redshift cluster.

Answer

The S3 bucket uses SSE-KMS encryption and the role lacks kms:Decrypt.

Answer

The IAM role name contains a typo in the COPY command.

Question 14

A company runs a data lake on Amazon S3 with AWS Glue for ETL. The data is stored in Parquet format and partitioned by date. The data engineer notices that queries using Amazon Athena are scanning large amounts of data even when filtering on the partition column. Which TWO actions would improve query performance? (Choose TWO)

Accepted Answer

Ensure that the WHERE clause uses the partition column correctly. Partition pruning requires the WHERE clause to filter on the partition column to reduce data scanned. Option E is correct because enabling predicate pushdown in Athena allows the query engine to push filtering conditions down to the data source, further reducing the amount of data scanned. Option A is incorrect because while Avro is a row-oriented format, it is not more efficient for analytics than Parquet; Parquet is columnar and better for selective queries. Option C is incorrect because CSV is not compressed and would increase data scanned. Option D is incorrect because simply increasing the number of partitions without proper filtering does not improve performance; excessive partitions can even degrade performance due to metadata overhead.

Answer

Use a different file format like Avro

Answer

Convert the data from Parquet to CSV for better compression

Answer

Increase the number of partitions by adding a second partition column

Question 15

A data pipeline uses AWS Lambda to process records from an Amazon Kinesis Data Stream. The Lambda function is idempotent and runs once per record. Recently, the function started failing with 'ProvisionedThroughputExceededException' when writing to a DynamoDB table. Which action should the data engineer take to resolve this?

Accepted Answer

Implement retry logic with exponential backoff in the Lambda function.. Implementing retry logic with exponential backoff allows the Lambda function to handle transient 'ProvisionedThroughputExceededException' errors by retrying write operations with increasing delays, which reduces the load on DynamoDB and gives the table capacity to recover. Since the function is idempotent, retries are safe. Option A is incorrect: decreasing the batch size reduces the number of records per invocation but does not directly address DynamoDB throttling; it may even increase the number of concurrent invocations, potentially worsening the issue. Option B is incorrect: increasing reserved concurrency allows more Lambda functions to run concurrently, which would increase the write rate to DynamoDB, exacerbating throttling. Option D is incorrect: increasing Kinesis shards increases the stream's ingestion capacity but does not affect DynamoDB's throughput limits; the bottleneck is at the DynamoDB table, not the stream.

Answer

Decrease the Lambda function's batch size to process fewer records per invocation.

Answer

Increase the Lambda function's reserved concurrency.

Answer

Increase the number of shards in the Kinesis stream.