DEA-C01 Data Operations and Support Practice Test 20 — 15 Questions

Question 1

A data engineer is designing an ETL pipeline that uses AWS Glue to process data from an Amazon DynamoDB table and write results to an S3 bucket in Parquet format. The pipeline must handle schema changes in the source DynamoDB table. Which THREE steps should the engineer take to ensure the pipeline handles schema evolution? (Choose THREE.)

Accepted Answer

Use Glue's 'recast' transformation to handle type changes.. Options A, B, and D are correct. Updating the Glue Data Catalog allows the crawler to update schema. 'mergeSchema' is a Spark option that merges schemas. 'recast' option in Glue helps handle type changes. Option C is wrong because deleting partitions is not related to schema evolution. Option E is wrong because converting to CSV is not a schema evolution strategy.

Answer

Convert the Parquet output to CSV to avoid schema constraints.

Answer

Partition the data by date and delete old partitions.

Question 2

A data engineer needs to transfer 50 TB of data from an on-premises Hadoop cluster to Amazon S3. The network bandwidth is limited to 500 Mbps. Which TWO methods are appropriate for this transfer? (Choose TWO.)

Accepted Answer

Order an AWS Snowball Edge device to physically ship the data.. Options A and C are correct. AWS Snowball Edge is a physical device for large data transfers over slow networks. AWS DataSync can transfer data over the network with optimization. Option B is wrong because S3 Transfer Acceleration speeds up transfers but still requires network bandwidth. Option D is wrong because AWS Direct Connect is a dedicated network connection, not a transfer method. Option E is wrong because Amazon Kinesis is for streaming data, not bulk transfer.

Answer

Set up an AWS Direct Connect connection for higher bandwidth.

Answer

Use S3 Transfer Acceleration to upload over the internet.

Answer

Use Amazon Kinesis Data Firehose to stream the data.

Question 3

A company uses Amazon Redshift for its data warehouse. During a routine audit, the data engineer discovers that some queries are returning stale data even though the underlying source data has been updated. The engineer confirms that the COPY command completes successfully and that no errors are reported. Which action should the engineer take to ensure queries reflect the latest data?

Accepted Answer

Refresh the materialized views that the queries are using.. Option B is correct because Redshift does not automatically maintain materialized views; they need to be refreshed to reflect changes in the base tables. Option A is wrong because Redshift does not have a cache that needs clearing in this context. Option C is wrong because VACUUM reclaims space and sorts tables, but does not refresh materialized views. Option D is wrong because ANALYZE updates statistics, not the data content.

Answer

Run the VACUUM command on the source tables.

Answer

Clear the Redshift result cache by running RESET ALL.

Answer

Run the ANALYZE command on the source tables.

Question 4

A data engineer at a financial services company manages an AWS Glue ETL pipeline that processes transaction data from Amazon S3 to Amazon Redshift for reporting. The pipeline runs every hour and uses a Glue job that reads Parquet files, performs transformations in Spark, and writes to Redshift using the JDBC connector. Recently, the job has been failing intermittently with the error: 'java.sql.BatchUpdateException: ERROR: null value in column "transaction_id" violates not-null constraint'. The data engineer has verified that the source Parquet files do contain non-null values for transaction_id. The job uses a DynamicFrame and applies a mapping to rename columns. The engineer also noticed that the failure occurs only during peak hours when there is high concurrency on Redshift. Which course of action should the engineer take to resolve this issue?

Accepted Answer

Review the Glue job's mapping transformation to ensure transaction_id is correctly mapped and not dropped.. Option C is correct. The error suggests that some rows are being written with null transaction_id. During high concurrency, Redshift might be rejecting the batch due to a transient issue, but the error is about null constraint. The most likely cause is that the mapping is incorrectly dropping or nullifying the column. Option A is wrong because increasing Glue's worker count does not address the null value issue. Option B is wrong because increasing Redshift WLM concurrency could exacerbate the problem. Option D is wrong because the source files are not the issue.

Answer

Add a filter in Glue to remove rows with null transaction_id.

Answer

Increase the Redshift WLM concurrency scaling to handle more queries.

Answer

Increase the number of Glue workers to handle peak-hour load.

Question 5

A data engineer notices that an Amazon Kinesis Data Firehose delivery stream is failing to deliver data to an Amazon S3 bucket. The engineer verifies that the S3 bucket exists and that the IAM role attached to the delivery stream has the necessary permissions. What is the MOST likely cause of the failure?

Accepted Answer

No data is being written to the Kinesis Data Firehose delivery stream.. Option C is correct because if no data is written to the stream, Firehose has nothing to deliver. Option A is wrong because delivery streams typically use S3 as a destination, not Kinesis Data Streams. Option B is wrong because insufficient permissions would cause an access denied error. Option D is wrong because CloudWatch Logs is for monitoring, not for storing delivery data.

Answer

The delivery stream is configured to deliver to Amazon CloudWatch Logs.

Answer

The IAM role does not have permissions to write to the S3 bucket.

Answer

The delivery stream is configured to deliver to Amazon Kinesis Data Streams instead of S3.

Question 6

A data engineer is responsible for a real-time data pipeline that ingests clickstream data from a website into Amazon Kinesis Data Streams, then processed by an AWS Lambda function that writes to an Amazon DynamoDB table for user session tracking. The Lambda function is idempotent and uses the DynamoDB PutItem API with a condition expression to avoid overwriting existing records. Over the past week, the engineer has observed an increase in DynamoDB write throttling (ProvisionedThroughputExceededException) during peak traffic hours. The DynamoDB table has on-demand capacity. The engineer checks the Lambda function's reserved concurrency and finds it set to 1000. The Kinesis stream has 10 shards. The Lambda function's batch size is set to 100. The engineer suspects that the retry behavior is causing duplicate writes and throttling. Which change should the engineer make to reduce throttling?

Accepted Answer

Decrease the Lambda batch size to 10 to reduce the number of records processed per invocation.. Option B is correct. On-demand DynamoDB can scale, but it has a per-partition throughput limit. Reducing the Lambda batch size reduces the number of concurrent writes per shard, decreasing the chance of hitting partition limits. Option A is wrong because increasing shards would increase concurrency, worsening throttling. Option C is wrong because decreasing reserved concurrency could cause Lambda throttling but not DynamoDB throttling. Option D is wrong because using a DynamoDB stream adds complexity and does not directly reduce write throttling.

Answer

Increase the number of Kinesis shards to 20 to distribute the load.

Answer

Decrease the Lambda reserved concurrency to 500 to limit the number of concurrent invocations.

Answer

Use a DynamoDB Stream to trigger a second Lambda function that writes to the table.

Question 7

A company runs an Amazon EMR cluster with Spark jobs that process data from Amazon S3. The data engineer receives an alert that one of the Spark jobs failed with an OutOfMemoryError. The job processes large files and uses the default Spark configurations. Which configuration change is MOST likely to resolve the issue?

Accepted Answer

Increase the spark.executor.memory configuration.. Option D is correct because increasing spark.executor.memory gives each executor more memory to handle large data processing. Option A is wrong because reducing the number of cores would reduce parallelism, potentially worsening the problem. Option B is wrong because dynamic allocation is enabled by default and helps, but the issue is executor memory. Option C is wrong because increasing the number of executors without increasing memory per executor does not address the OOM per executor.

Answer

Increase the number of executors.

Answer

Disable dynamic resource allocation.

Answer

Decrease the number of cores per executor.

Question 8

A data engineer is troubleshooting an AWS Glue ETL job that uses a Python shell script to extract data from an Amazon RDS for PostgreSQL database and load it into an Amazon Redshift table. The job runs successfully, but the data engineer notices that the row count in Redshift is consistently lower than the row count in PostgreSQL. The job uses a SELECT * query without any filtering. The data engineer suspects that some rows are being dropped during the transfer. The job uses the psycopg2 library to connect to PostgreSQL and the psycopg2 connection is configured with autocommit=True. The Redshift table has no constraints that would reject rows. What is the most likely cause of the missing rows?

Accepted Answer

The SELECT * query includes columns with data types that are not supported by psycopg2.. Option C is correct. If the Redshift table is set to distribute data, and if the distribution key is not unique, some rows may be lost if there are duplicates? Actually, the most common cause is that the Glue job is using multiple executors, and the data is being split across them, but Python shell uses a single executor. However, the most plausible answer is that the SELECT * query may include columns with special characters that are not handled correctly. Option A is wrong because autocommit=True should not cause data loss. Option B is wrong because SSL/TLS is about encryption, not row count. Option D is wrong because the Redshift table has no constraints.

Answer

The SSL/TLS connection to PostgreSQL is dropping packets.

Answer

The autocommit=True setting is causing incomplete transactions.

Answer

The Redshift table has a distribution key that causes some rows to be silently discarded.

Question 9

A data engineer uses AWS Database Migration Service (DMS) to migrate an on-premises Oracle database to Amazon Aurora MySQL. The migration is successful, but the engineer notices that the target Aurora cluster has a higher CPU utilization than expected during the full load phase. What is the MOST likely cause?

Accepted Answer

The DMS task is configured to use multiple parallel threads to load data, overwhelming the target instance.. Option A is correct because DMS uses multiple tasks in parallel to maximize throughput, which can cause high CPU on the target. Option B is wrong because DMS does not use read replicas during full load. Option C is wrong because LOB settings affect column size handling, not CPU load. Option D is wrong because validation occurs after migration, not during full load.

Answer

The DMS task has LOB mode set to 'Full LOB mode', causing additional processing.

Answer

DMS is performing data validation during the full load phase.

Answer

DMS is reading from an Amazon Aurora read replica instead of the primary instance.

Question 10

A data engineer configured an AWS Glue job that reads from an S3 bucket and writes to an Amazon Redshift table. The job runs successfully, but the data in Redshift is missing some records that exist in S3. The engineer suspects the issue may be related to the job's bookmarks. What should the engineer do to ensure all records are processed?

Accepted Answer

Disable job bookmarks in the Glue job configuration.. Option B is correct because disabling job bookmarks forces the Glue job to reprocess all data, which will include the missing records. Option A is wrong because increasing the number of workers does not address the bookmark issue. Option C is wrong because updating the IAM role does not affect bookmarks. Option D is wrong because job bookmarks do not have a 'reset' API; they are managed via the job configuration.

Answer

Update the IAM role to grant additional S3 read permissions.

Answer

Reset the job bookmark using the AWS Glue API.

Answer

Increase the number of workers in the Glue job.

Question 11

A data engineer needs to move data from an Amazon S3 bucket to an Amazon Redshift cluster on a daily schedule. The data is in CSV format and the target table already exists. Which AWS service should the engineer use to automate this task?

Accepted Answer

AWS Glue. Option B is correct because AWS Glue can be used to run ETL jobs that copy data from S3 to Redshift on a schedule. Option A is wrong because Amazon Athena is a query service, not an ETL scheduler. Option C is wrong because Amazon EMR is a big data platform but requires more setup for simple scheduling. Option D is wrong because Amazon Kinesis Data Analytics is for real-time streaming data.

Answer

Amazon Athena

Answer

Amazon EMR

Answer

Amazon Kinesis Data Analytics

Question 12

A data engineer is troubleshooting a Kinesis Data Analytics application that processes streaming data. The application is falling behind, and the metric 'MillisBehindLatest' is consistently above 60000. The source Kinesis stream has 10 shards, and the application uses a Flink application with default parallelism. What is the MOST likely cause of the lag?

Accepted Answer

The Flink application parallelism is set to 1.. Option C is correct because default parallelism in Flink is 1, which means only one task processes data from all shards, causing a bottleneck. Option A is wrong because increasing shards would increase parallelism but the issue is with parallelism. Option B is wrong because the retention period does not affect lag. Option D is wrong because the stream is the source, not the sink.

Answer

The sink (destination) is throttling writes.

Answer

The Kinesis stream has too few shards.

Answer

The retention period of the Kinesis stream is too short.

Question 13

A company uses Amazon S3 to store sensitive data. The data engineer needs to ensure that all data in transit between the S3 bucket and clients is encrypted. Which configuration should the engineer implement?

Accepted Answer

Create an S3 bucket policy that denies requests where SecureTransport is false.. Option D is correct because a bucket policy that denies requests without the aws:SecureTransport condition enforces HTTPS for all access. Option A is wrong because S3 default encryption only encrypts data at rest, not in transit. Option B is wrong because CloudFront does not enforce HTTPS by default; it can be configured but is not the direct solution. Option C is wrong because SSE-C is server-side encryption at rest, not in transit.

Answer

Use Amazon CloudFront to serve the content and enable SSL.

Answer

Enable default encryption on the S3 bucket using SSE-S3.

Answer

Use SSE-C to encrypt the data with a customer-provided key.

Question 14

Refer to the exhibit. An IAM policy is attached to an IAM user. The user is trying to upload an object to 's3://data-lake-bucket/confidential/report.pdf' using the AWS CLI. The upload fails with an AccessDenied error. What is the reason for the failure?

Accepted Answer

An explicit Deny statement overrides the Allow statement for the 'confidential/' prefix.. Option A is correct because an explicit Deny overrides any Allow. The Deny statement blocks all s3 actions on the confidential prefix, even though the Allow statement grants PutObject. Option B is wrong because the policy allows PutObject on the bucket. Option C is wrong because the resource is specified correctly. Option D is wrong because the user has permissions on other parts of the bucket.

Answer

The policy does not include 's3:PutObject' action.

Answer

The resource ARN in the Allow statement does not cover the specific object.

Answer

The user does not have permission to access the bucket at all.

Question 15

A company runs a data pipeline that uses AWS Lambda to process files uploaded to an S3 bucket. Recently, some files have been processed multiple times. The Lambda function is triggered by S3 event notifications. What is the MOST likely cause of duplicate processing?

Accepted Answer

S3 event notifications are delivered at least once.. Option C is correct because S3 event notifications are delivered at least once, and can be delivered more than once in rare cases. Option A is wrong because Lambda does not have a retry mechanism upon failure that causes duplicates; it would retry the same invocation, but not create duplicates on success. Option B is wrong because the function should be idempotent, but the question asks for the cause. Option D is wrong because Lambda concurrency does not cause duplicates.

Answer

The Lambda function has a high error rate and retries.

Answer

The Lambda function is not idempotent.

Answer

The Lambda function has a reserved concurrency setting.