DEA-C01 Data Operations and Support Practice Test 21 — 15 Questions

Question 1

A company runs an Amazon EMR cluster with Spark jobs that process data from Amazon S3. The data engineer receives an alert that one of the Spark jobs failed with an OutOfMemoryError. The job processes large files and uses the default Spark configurations. Which configuration change is MOST likely to resolve the issue?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Increase the spark.executor.memory configuration.. Increasing spark.executor.memory allocates more memory per executor, directly addressing the OutOfMemoryError when processing large files. Option B (increasing executors) does not increase memory per executor, so each executor remains susceptible to OOM. Option C (disabling dynamic resource allocation) would prevent the cluster from adding resources dynamically, potentially worsening the situation. Option D (decreasing cores per executor) reduces parallelism but does not increase memory per executor; the OOM occurs because each executor lacks sufficient memory, not because of too many cores.

Answer

Increase the number of executors.

Answer

Disable dynamic resource allocation.

Answer

Decrease the number of cores per executor.

Question 2

A data engineer is troubleshooting an AWS Glue ETL job that uses a Python shell script to extract data from an Amazon RDS for PostgreSQL database and load it into an Amazon Redshift table. The job runs successfully, but the data engineer notices that the row count in Redshift is consistently lower than the row count in PostgreSQL. The job uses a SELECT * query without any filtering. The data engineer suspects that some rows are being dropped during the transfer. The job uses the psycopg2 library to connect to PostgreSQL and the psycopg2 connection is configured with autocommit=True. The Redshift table has no constraints that would reject rows. What is the most likely cause of the missing rows?

Accepted Answer

The SELECT * query includes columns with data types that are not supported by psycopg2.. The SELECT * query may include columns with data types (e.g., arrays, hstore, or geometric types) that psycopg2 does not fully support, causing those columns to be read as NULL or silently skipped during extraction. This would result in fewer rows being inserted into Redshift if the unsupported columns are part of a unique constraint or if NULL handling causes row drops. Option B is incorrect because SSL/TLS packet drops would cause connection errors, not consistent missing rows. Option C is incorrect because autocommit=True ensures each query is a standalone transaction, preventing incomplete transactions. Option D is incorrect because the problem states the Redshift table has no constraints that would reject rows, and distribution keys do not silently discard rows.

Answer

The SSL/TLS connection to PostgreSQL is dropping packets.

Answer

The autocommit=True setting is causing incomplete transactions.

Answer

The Redshift table has a distribution key that causes some rows to be silently discarded.

Question 3

A data engineer uses AWS Database Migration Service (DMS) to migrate an on-premises Oracle database to Amazon Aurora MySQL. The migration is successful, but the engineer notices that the target Aurora cluster has a higher CPU utilization than expected during the full load phase. What is the MOST likely cause?

Accepted Answer

The DMS task is configured to use multiple parallel threads to load data, overwhelming the target instance.. During full load, DMS uses multiple parallel threads (by default up to 4 or more) to maximize throughput, which can overwhelm the target Aurora cluster's CPU. Option A is incorrect: 'Full LOB mode' affects how large objects are handled and may increase latency, but it is not the primary cause of high CPU utilization. Option B is incorrect: DMS performs data validation after the full load phase, not during it. Option C is incorrect: DMS reads from the source (on-premises Oracle), not from an Aurora read replica; moreover, read replicas are not used for writing.

Answer

The DMS task has LOB mode set to 'Full LOB mode', causing additional processing.

Answer

DMS is performing data validation during the full load phase.

Answer

DMS is reading from an Amazon Aurora read replica instead of the primary instance.

Question 4

A data engineer needs to move data from an Amazon S3 bucket to an Amazon Redshift cluster on a daily schedule. The data is in CSV format and the target table already exists. Which AWS service should the engineer use to automate this task?

Accepted Answer

AWS Glue. AWS Glue is the correct choice because it provides managed ETL (extract, transform, load) capabilities and can be scheduled to run daily. Glue jobs can directly read CSV files from S3 and write to a Redshift table that already exists. Option A (correct) accurately identifies AWS Glue for this automated data movement task. Option B (Amazon Athena) is wrong because Athena is an interactive query service, not an ETL scheduler; it cannot autonomously move data on a schedule. Option C (Amazon EMR) is wrong because while EMR can handle such tasks, it requires provisioning a cluster, managing lifecycle, and more configuration than needed for a simple daily copy—Glue is simpler and more cost-effective for this use case. Option D (Amazon Kinesis Data Analytics) is wrong because it is designed for processing streaming data in real time, not for batch copying of CSV files.

Answer

Amazon Athena

Answer

Amazon EMR

Answer

Amazon Kinesis Data Analytics

Question 5

A data engineer is troubleshooting a Kinesis Data Analytics application that processes streaming data. The application is falling behind, and the metric 'MillisBehindLatest' is consistently above 60000. The source Kinesis stream has 10 shards, and the application uses a Flink application with default parallelism. What is the MOST likely cause of the lag?

Accepted Answer

The Flink application parallelism is set to 1.. In Kinesis Data Analytics with Flink, the default parallelism is 1. With 10 shards in the source Kinesis stream, a single parallel task must read from all shards, creating a bottleneck and causing the 'MillisBehindLatest' metric to be consistently high (above 60000). Option A is wrong because while a throttling sink can cause backpressure, the question specifically states the application is falling behind and the metric is high, which is more directly explained by insufficient parallelism. Option C is wrong because 10 shards is typically sufficient; the issue is how they are consumed. Option D is wrong because the retention period does not affect the lag metric; it only controls how long data is stored.

Answer

The sink (destination) is throttling writes.

Answer

The Kinesis stream has too few shards.

Answer

The retention period of the Kinesis stream is too short.

Question 6

A company uses Amazon S3 to store sensitive data. The data engineer needs to ensure that all data in transit between the S3 bucket and clients is encrypted. Which configuration should the engineer implement?

Accepted Answer

Create an S3 bucket policy that denies requests where SecureTransport is false.. An S3 bucket policy that denies requests where SecureTransport is false enforces HTTPS for all access, encrypting data in transit. Option A is wrong because CloudFront with SSL can enforce HTTPS but is not a direct S3 configuration and may incur additional costs. Option B is wrong because SSE-S3 only encrypts data at rest, not in transit. Option D is wrong because SSE-C also encrypts data at rest with a customer-provided key, not in transit.

Answer

Use Amazon CloudFront to serve the content and enable SSL.

Answer

Enable default encryption on the S3 bucket using SSE-S3.

Answer

Use SSE-C to encrypt the data with a customer-provided key.

Question 7

Refer to the exhibit. An IAM policy is attached to an IAM user. The user is trying to upload an object to 's3://data-lake-bucket/confidential/report.pdf' using the AWS CLI. The upload fails with an AccessDenied error. What is the reason for the failure?

Accepted Answer

An explicit Deny statement overrides the Allow statement for the 'confidential/' prefix.. The IAM policy includes an explicit Deny statement that denies all s3 actions on the 'confidential/' prefix. Even though there is an Allow statement that grants s3:PutObject on the bucket, the explicit Deny overrides it, causing the upload to fail with AccessDenied. Option A is incorrect because the policy does include the s3:PutObject action. Option B is incorrect because the resource ARN in the Allow statement covers the bucket and objects, but the Deny specifically targets 'confidential/'. Option C is incorrect because the user does have permission to access the bucket via the Allow statement, but the Deny blocks access to the specific object under 'confidential/'.

Answer

The policy does not include 's3:PutObject' action.

Answer

The resource ARN in the Allow statement does not cover the specific object.

Answer

The user does not have permission to access the bucket at all.

Question 8

A company runs a data pipeline that uses AWS Lambda to process files uploaded to an S3 bucket. Recently, some files have been processed multiple times. The Lambda function is triggered by S3 event notifications. What is the MOST likely cause of duplicate processing?

Accepted Answer

S3 event notifications are delivered at least once.. S3 event notifications are delivered at least once, meaning they can be sent multiple times, causing the Lambda function to be invoked repeatedly for the same file. This is the most likely cause of duplicate processing. Option A is incorrect: while Lambda retries on error for asynchronous invocations, it retries the same failed invocation, not creating multiple invocations for a successful event. Option B is incorrect: lack of idempotency does not cause duplicates; it simply means the function might not handle duplicates properly, but the root cause is the duplicate trigger. Option C is incorrect: reserved concurrency limits the number of concurrent executions but does not generate duplicate invocations.

Answer

The Lambda function has a high error rate and retries.

Answer

The Lambda function is not idempotent.

Answer

The Lambda function has a reserved concurrency setting.

Question 9

A data engineer is monitoring an Amazon Kinesis Data Streams application that processes real-time events. The application uses a Kinesis Client Library (KCL) consumer. The engineer notices that the consumer is lagging behind the producer, and the lag is increasing over time. The stream has 10 shards. Which action will MOST effectively reduce the lag?

Accepted Answer

Increase the number of shards in the Kinesis data stream.. Increase the number of shards in the Kinesis data stream. Increasing shards increases the stream's read capacity and allows more parallel consumers to process data, which reduces lag. Option A is incorrect because using multiple KCL workers per shard is not recommended; each shard should be processed by one worker to avoid duplicate processing. Option C is incorrect because decreasing the number of records per shard per second would reduce throughput, not help catch up. Option D is incorrect because decreasing shards reduces capacity, worsening lag.

Answer

Use multiple KCL workers per shard to increase processing capacity.

Answer

Decrease the number of records per shard per second.

Answer

Decrease the number of shards in the Kinesis data stream.

Question 10

A data engineer needs to set up a data catalog for a new data lake in AWS Glue. The data resides in S3 in Parquet format. The engineer wants to ensure that the schema is automatically detected and updated when new columns are added to the data. Which configuration should the engineer use?

Accepted Answer

Configure the crawler's 'Schema updates' option to 'Update the table schema'.. Configuring the crawler's 'Schema updates' option to 'Update the table schema' allows Glue crawlers to automatically detect and update the schema when new columns are added to the data. Option A is wrong because a partition index is used to improve query performance on partitioned data, not to update the schema. Option C is wrong because setting the crawler's database output to a new database does not affect schema updates; it simply directs the crawler to write to a different database. Option D is wrong because partition indexing (enable partition indexing) is about indexing partitions for faster querying, not about schema updates.

Answer

Add a partition index to the Glue Data Catalog table.

Answer

Set the crawler's 'Database' output to a new database.

Answer

Enable partition indexing on the table.

Question 11

A data engineer has set up an AWS Lambda function that processes files uploaded to an S3 bucket. The function is triggered by S3 event notifications. However, the function is not being invoked when a file is uploaded. The engineer checks the Lambda function's CloudWatch Logs and finds no execution logs. What should the engineer check FIRST?

Accepted Answer

Verify that the S3 bucket has an event notification configured for the Lambda function.. The correct first step is to verify the S3 event notification configuration (Option C). Since there are no execution logs in CloudWatch, the Lambda function is not being triggered at all. This indicates a problem with the trigger mechanism, not with the function's code, permissions, or network configuration. Option A (checking code) is premature because code errors would appear in logs after invocation. Option B (IAM role) affects what the function can do during execution, not whether it is invoked. Option D (VPC) affects network access but does not prevent triggering. Therefore, the S3 bucket's event notification must be checked to ensure it is correctly set up to invoke the Lambda function.

Answer

Check the Lambda function's code for errors.

Answer

Verify that the Lambda function's IAM role has permissions to read from S3.

Answer

Check if the Lambda function is attached to a VPC.

Question 12

A company uses Amazon EMR to run Spark jobs on a transient cluster. The jobs are submitted via a step in the cluster. The cluster is configured to auto-terminate after the last step completes. However, the cluster is not terminating even though the step shows as 'COMPLETED'. What could be the cause?

Accepted Answer

The step's 'ActionOnFailure' parameter is set to 'CONTINUE' and 'KeepClusterAliveOnFailure' is true.. In Amazon EMR, the step-level parameter `KeepClusterAliveOnFailure` (or `KeepJobFlowAliveWhenNoSteps` for the cluster) can prevent auto-termination. When `ActionOnFailure` is set to `CONTINUE` and `KeepClusterAliveOnFailure` is `true`, the cluster remains running even after the last step completes, overriding the transient cluster's auto-terminate setting. Option A is incorrect because root device size does not affect termination. Option B is incorrect because the step status 'COMPLETED' indicates success, not a bug. Option C is incorrect because the cluster is explicitly configured as transient and auto-terminating, not long-running.

Answer

The cluster's root device size is too large.

Answer

The step failed with an error, but the status shows 'COMPLETED' due to a reporting bug.

Answer

The cluster is configured as a long-running cluster.

Question 13

A data engineer is designing a data pipeline that ingests streaming data from an IoT device fleet. The data must be processed in near real-time and stored in Amazon S3 for long-term analytics. Which TWO AWS services should the engineer use together to achieve this?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Streams (Option D) enables real-time ingestion of streaming data from IoT devices. Amazon Kinesis Data Firehose (Option C) can consume data from a Kinesis Data Stream and deliver it to Amazon S3 for long-term analytics. Option A (Athena) is a query service, not an ingestion or delivery service. Option B (AWS Glue) is designed for batch ETL, not real-time streaming. Option E (SQS) is a message queue service, not optimized for real-time streaming and delivery to S3.

Answer

Amazon Athena

Answer

AWS Glue

Answer

Amazon Simple Queue Service (SQS)

Question 14

A data engineer is monitoring an Amazon Redshift cluster and notices that the disk space usage is increasing rapidly. The engineer wants to reclaim space from deleted rows. Which command should the engineer run?

Accepted Answer

VACUUM. The VACUUM command in Amazon Redshift reclaims disk space from deleted rows and reorganizes the data to improve query performance. Option B is wrong because ANALYZE updates statistics for the query optimizer, not reclaims space. Option C is wrong because UNLOAD exports data from the cluster to Amazon S3. Option D is wrong because COPY loads data from files into the cluster.

Answer

ANALYZE

Answer

UNLOAD

Answer

COPY

Question 15

A data engineer is troubleshooting a slow-running Amazon Redshift query. The query joins several large tables and performs aggregations. The engineer runs EXPLAIN and sees a 'DS_DIST_ALL' step. Which TWO actions will MOST likely improve query performance? (Choose TWO.)

Accepted Answer

Change the distribution style of the tables to DISTSTYLE KEY on the join columns.. The DS_DIST_ALL step in the query plan indicates that data is being broadcast from one node to all others, causing significant network overhead. Option C is correct because changing the distribution style to DISTSTYLE KEY on the join columns ensures that matching rows are co-located on the same node, reducing the need for redistribution. Option E is correct because defining appropriate SORTKEYs based on query predicates allows the query optimizer to use zone maps to skip irrelevant blocks, speeding up scans and aggregations. Option A is incorrect; VACUUM reorganizes data on disk but does not affect distribution. Option B is incorrect; CNAME is a DNS record type, not a Redshift command. Option D is incorrect; increasing nodes might not directly fix the distribution issue and is less targeted than changing distribution style.

Answer

Run the VACUUM command on all tables.

Answer

Use the CNAME command to rename the tables.

Answer

Increase the number of nodes in the Redshift cluster.