DEA-C01 Practice Exam 70 — 20 Questions

Question 1

A data engineer is monitoring an AWS Glue ETL job that intermittently fails with 'Container killed by YARN for exceeding memory limits' during a large shuffle stage. The job reads from Amazon S3, performs a groupByKey aggregation, and writes to Amazon S3. The engineer wants to reduce the chance of executor memory exhaustion without changing the source data. (Choose two.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Replace the groupByKey operation with reduceByKey or aggregateByKey to combine values before the shuffle.. The container kill originates from a single executor holding too much data during the shuffle. Adding workers spreads the shuffle across more containers, reducing per-executor memory demand. Replacing groupByKey with reduceByKey or aggregateByKey introduces map-side combining so far less data crosses the network. Both changes reduce peak memory in the shuffle stage without touching the source data.

Answer

Enable the Glue job bookmark to skip previously processed files.

Answer

Change the output write format from Parquet to uncompressed JSON.

Answer

Set the job's max concurrent runs to 1 to avoid overlapping executions.

Question 2

A data engineer is using AWS Database Migration Service (AWS DMS) to migrate an on-premises Oracle database to Amazon Aurora PostgreSQL. The migration is ongoing, and the engineer notices that some transactions are not being replicated to the target. The DMS task is configured for full load plus change data capture (CDC). Which action should the engineer take to troubleshoot the missing transactions?

Accepted Answer

Check the DMS task's CloudWatch logs for errors related to unsupported data types or DDL changes.. When AWS DMS CDC skips transactions, it is often due to errors such as unsupported data types or DDL changes. These errors are logged in CloudWatch Logs. Reviewing the logs allows the engineer to identify the specific cause and take corrective action, such as excluding the problematic table or adjusting the task settings. This is the most direct troubleshooting step.

Answer

Enable AWS DMS validation to compare source and target rows and identify discrepancies.

Answer

Modify the DMS task to use 'Drop and create' target table preparation mode.

Answer

Increase the DMS replication instance size to handle the transaction volume.

Question 3

A data engineer needs to run a one-time transformation on a 500 GB dataset stored in Amazon S3. The transformation is written in Python and uses pandas, which cannot handle the full dataset in memory. The engineer wants a serverless option that can parallelize the work without managing servers. Which AWS service should the engineer use?

Accepted Answer

Amazon EMR Serverless with a Spark job that uses Spark DataFrame operations instead of pandas.. EMR Serverless provides a serverless Spark environment that can distribute a large transformation across workers, removing the single-node memory limit of pandas. By converting the logic to Spark DataFrames, the engineer can process 500 GB in parallel without provisioning or managing servers, which matches the stated constraints.

Answer

AWS Lambda with a function that streams the S3 objects and processes them in 15-minute increments.

Answer

AWS Glue with a Python shell job and the pandas library, splitting the data into chunks.

Answer

Amazon Athena with a CREATE TABLE AS SELECT statement that applies the transformation in SQL.

Question 4

A data engineer manages an Amazon S3 data lake where analytics queries run through Amazon Athena. Monthly partition folders hold Parquet files, and each partition contains tens of thousands of small files averaging 40 KB. Athena queries that scan a single month take much longer than expected and consume far more bytes scanned than the actual data volume. The engineer must improve query performance without changing the table schema or the folder layout. What should the engineer do?

Accepted Answer

Run an AWS Glue ETL job that reads each partition and rewrites it as fewer, larger Parquet files of roughly 128 MB, then update the partitions in the Data Catalog.. Athena performance degrades when a partition holds a huge number of very small objects because each object requires a separate request and contributes metadata overhead, inflating both runtime and reported bytes scanned. Compacting the files with an AWS Glue job into larger Parquet objects preserves the schema and partition layout while cutting that overhead, which is the standard remedy for this pattern.

Answer

Enable S3 Transfer Acceleration on the bucket so that Athena can retrieve the small objects with lower latency across edge locations.

Answer

Attach an S3 Lifecycle policy that transitions the Parquet objects to S3 Glacier Instant Retrieval after 30 days to improve read throughput.

Answer

Increase the number of partitions by splitting each monthly folder into daily folders and repointing the table at the new prefixes.

Question 5

A data engineer is managing an Amazon Redshift cluster that experiences performance degradation during peak query loads. The cluster uses a dc2.large node type with 2 nodes. Analysis shows that the workload involves frequent large table scans and complex joins. The engineer wants to improve query performance without changing the overall data volume. Which action should the engineer take?

Accepted Answer

Change the distribution style of large tables to KEY on the join columns and enable sort keys on frequently filtered columns.. Proper distribution and sort keys are fundamental Redshift tuning techniques. Distributing large tables on join keys collocates data, minimizing network traffic during joins. Sort keys on filter columns allow the query engine to skip blocks. These changes directly address the performance bottlenecks without adding hardware, making them the most effective and cost-efficient solution.

Answer

Increase the number of nodes in the cluster by resizing to a larger node type with more memory.

Answer

Enable concurrency scaling to automatically add transient clusters during peak loads.

Answer

Enable short query acceleration (SQA) to prioritize short-running queries.

Question 6

A data engineer manages an Amazon S3 data lake with millions of small JSON files under prefixes partitioned by year/month/day. Amazon Athena queries scan far more data than expected and return slowly. The engineer wants to reduce bytes scanned and improve query performance while keeping files in S3 and queryable by Athena. Which solution meets these requirements with the LEAST operational overhead?

Accepted Answer

Use AWS Glue to convert the JSON files to Apache Parquet, write them to partitioned prefixes, and register the resulting tables in the AWS Glue Data Catalog.. Athena charges and performs based on bytes scanned, and row-oriented JSON forces reading entire objects. Converting to columnar Parquet lets the engine read only referenced columns, while partitioning by year/month/day enables partition pruning so irrelevant dates are skipped. Registering the converted data in the AWS Glue Data Catalog makes it available to Athena with minimal management, directly reducing scanned bytes and latency.

Answer

Enable S3 Transfer Acceleration on the bucket and configure Athena workgroups to use it for all queries.

Answer

Attach an S3 Lifecycle policy that transitions the JSON objects to S3 Glacier Instant Retrieval after 30 days and query them through Athena.

Answer

Create an Amazon S3 Inventory report for each partition and use it to rewrite the Athena queries so they reference only the inventory files.

Question 7

A data engineer is using AWS Glue Studio to build a job that reads from an Amazon S3 bucket containing nested JSON files. The job must flatten the nested structure and write the output to Amazon Redshift. The engineer notices that the job fails with an error indicating that the Redshift table does not exist. The engineer has already created the Redshift cluster and database. What should the engineer do to resolve the error?

Accepted Answer

Create the target table in Amazon Redshift with the appropriate schema before running the job.. When writing to Amazon Redshift from AWS Glue, the target table must already exist. Glue does not automatically create Redshift tables. The engineer should create the table with the correct schema in Redshift, then rerun the job. The other transforms mentioned are for data manipulation within the job and do not address the missing table.

Answer

Add a 'ResolveChoice' transform before the Redshift writer to handle nested types.

Answer

Use the 'ApplyMapping' transform to map columns to Redshift data types.

Answer

Configure the Redshift writer to use the 'CREATE TABLE' option and specify the table name.

Question 8

A data engineer is building a data lake on Amazon S3. The raw data arrives as JSON files, but the analytics team needs to query the data using standard SQL in Amazon Athena with optimal performance and minimal cost. The engineer wants to convert the JSON to a columnar format that supports predicate pushdown and is natively supported by Athena. Which storage format should the engineer choose?

Accepted Answer

Apache Parquet. Apache Parquet is the best choice because it is a columnar format that allows Athena to read only the necessary columns, reducing data scanned and cost. It also supports predicate pushdown, which filters data at the storage level. Other formats like Avro, JSON, and CSV are row-based and less efficient for analytical queries in Athena.

Answer

CSV

Answer

JSON

Answer

Apache Avro

Question 9

A data engineer maintains an Amazon S3 data lake with millions of small JSON objects. The engineer needs to improve query performance by reducing the number of objects and compressing them into a columnar format that Amazon Athena can query efficiently. The data must remain partitioned by date. Which solution should the engineer use?

Accepted Answer

Use AWS Glue ETL to read the JSON objects, repartition by date, write them as Parquet with Snappy compression, and register the resulting tables in the AWS Glue Data Catalog.. Converting small JSON files to Parquet with Snappy compression and repartitioning by date reduces storage footprint and the amount of data scanned by Athena. AWS Glue ETL can perform this transformation and update the Data Catalog so Athena queries the new tables. This is the standard pattern for optimizing S3 data lakes for analytics.

Answer

Enable S3 Transfer Acceleration on the bucket and run Amazon Athena queries with the existing JSON objects.

Answer

Use Amazon S3 Lifecycle policies to transition the JSON objects to S3 Glacier Instant Retrieval and query them with Athena.

Answer

Create an Amazon EMR cluster and run a MapReduce job that merges the JSON files into larger JSON files without changing the format.

Question 10

A data engineer has an AWS Glue job that processes data from an Amazon S3 bucket and writes to an Amazon Redshift cluster. The job is scheduled to run daily. Recently, the job started failing with the error: 'java.sql.SQLException: [Amazon](500310) Invalid operation: Spectrum Scan Error: S3 Access Denied'. The engineer verifies that the IAM role associated with the Glue job has full access to the S3 bucket. What is the most likely cause of this error?

Accepted Answer

The Amazon Redshift cluster's IAM role lacks permission to access the S3 bucket.. The error occurs because Amazon Redshift, not AWS Glue, is attempting to access S3 during the COPY operation. The Redshift cluster's IAM role must have permissions to read the S3 bucket. Even if the Glue job's role has access, Redshift uses its own role. The engineer should attach a policy to the Redshift cluster's IAM role granting S3 read access.

Answer

The Glue job's script is using an incorrect S3 path for the Redshift COPY command.

Answer

The Amazon Redshift cluster is in a different AWS Region than the S3 bucket, causing access latency.

Answer

The S3 bucket policy explicitly denies access to the Glue job's IAM role.

Question 11

A data engineer needs to grant an AWS Glue ETL job access to read data from an Amazon S3 bucket that is encrypted with SSE-KMS using a customer managed key. The Glue job runs with an IAM role. Which action must the engineer take to allow the Glue job to decrypt the data?

Accepted Answer

Attach a policy to the Glue job's IAM role that allows kms:Decrypt and kms:GenerateDataKey on the specific KMS key.. To allow an AWS Glue job to read SSE-KMS encrypted data from S3, the IAM role associated with the Glue job must have permissions to use the KMS key for decryption. Specifically, the role needs kms:Decrypt and kms:GenerateDataKey permissions on the key. S3 bucket policies alone do not grant KMS access, and changing encryption to SSE-S3 or using access points does not satisfy the requirement to use SSE-KMS.

Answer

Enable default encryption on the S3 bucket using SSE-S3 instead of SSE-KMS, which eliminates the need for KMS permissions.

Answer

Use an S3 Access Point with a policy that allows the Glue job's IAM role to access objects, and configure the access point to use a different KMS key.

Answer

Modify the S3 bucket policy to allow the Glue job's IAM role to perform s3:GetObject, and rely on S3 to decrypt the data automatically.

Question 12

A data engineer needs to schedule a recurring AWS Glue ETL job that must run every night at 02:00 UTC and must not start a new run while a previous run is still executing. The engineer wants the simplest managed scheduling option that integrates natively with Glue job run state. Which approach should the engineer use?

Accepted Answer

Configure a Glue trigger of type SCHEDULED with a cron expression and set the job's maximum concurrency to 1.. Glue scheduled triggers use cron expressions to start jobs at fixed times, and the job's maximum concurrency setting controls whether overlapping runs are allowed. Setting maximum concurrency to 1 ensures that if a run is still active when the next trigger fires, the new run does not start. This keeps scheduling and concurrency control entirely within Glue, which is the simplest managed solution.

Answer

Create an AWS Step Functions state machine with a Wait state that loops every 24 hours and starts the Glue job.

Answer

Use an AWS Lambda function invoked by a CloudWatch Events rule to call StartJobRun on a fixed schedule.

Answer

Create an Amazon EventBridge scheduled rule with a cron expression that invokes the Glue job via a target.

Question 13

A media company ingests thousands of small JSON files per hour into an Amazon S3 bucket. A data engineer needs to convert these files into a compact, columnar format for efficient querying with Amazon Athena. The engineer wants to minimize storage costs and improve query performance. Which approach should the engineer take?

Accepted Answer

Use AWS Glue ETL to read the JSON files, convert them to Apache Parquet, and write the output to a new S3 prefix partitioned by date.. AWS Glue ETL is a fully managed extract, transform, and load service that can efficiently process large volumes of data. It can read JSON from S3, convert to Parquet, and write partitioned output. This reduces storage costs due to Parquet's compression and columnar format, and improves Athena query performance by reducing data scanned. Partitioning by date further optimizes queries.

Answer

Use Amazon Athena to create a new table as Parquet using CREATE TABLE AS SELECT (CTAS) from the JSON table.

Answer

Use Amazon Kinesis Data Firehose to convert JSON to Parquet and deliver to S3.

Answer

Use AWS Lambda to read each JSON file, convert to Parquet, and write back to S3.

Question 14

A data engineer needs to ingest data from an Amazon S3 bucket into an Amazon Redshift cluster. The data is stored as CSV files and is updated daily. The engineer wants to load only new data each day without duplicating existing records. Which AWS service or feature should the engineer use to automate this process?

Accepted Answer

AWS Glue with job bookmarks enabled to track processed files.. AWS Glue job bookmarks are designed to track processed data across job runs, enabling incremental processing. For daily loads from S3 to Redshift, a Glue ETL job with bookmarks ensures only new files are processed each day, preventing duplication. This automates the process and integrates with Redshift via the COPY command. Other options do not provide automatic incremental load tracking for S3 files.

Answer

Amazon Redshift Spectrum to query S3 data directly and insert new records.

Answer

Amazon Kinesis Data Firehose to stream S3 data to Redshift.

Answer

AWS Database Migration Service (DMS) with ongoing replication from S3 to Redshift.

Question 15

A data engineer is using AWS Glue to transform data from Amazon S3 and load it into Amazon Redshift. The engineer needs to improve the performance of the Glue job, which currently takes several hours to complete. The job reads large Parquet files, performs joins, and writes to Redshift. Which two actions should the engineer take to improve performance? (Choose two.)

Accepted Answer

Partition the source Parquet data in Amazon S3 by commonly filtered columns and use partition pruning in the Glue job.. Increasing DPUs adds compute resources to parallelize the job, and partitioning the source data enables partition pruning to reduce the amount of data read. Both actions directly address performance bottlenecks in a large Glue job. Other options either do not affect single-run performance or would degrade it.

Answer

Enable job bookmarks to avoid reprocessing previously processed data.

Answer

Convert the Parquet files to CSV to reduce storage size and speed up reading.

Answer

Use the 'Relationalize' transform to flatten the Parquet data before joining.

Question 16

A data engineer needs to monitor an AWS Glue ETL job that runs daily. The job sometimes fails due to missing partitions in the Data Catalog. The engineer wants to receive an alert when the job fails. What is the MOST operationally efficient way to achieve this?

Accepted Answer

Use AWS Glue job event notifications via Amazon EventBridge to trigger an AWS Lambda function that sends an Amazon SNS notification.. Amazon EventBridge can capture AWS Glue job state change events and route them to targets like Lambda or SNS. This provides immediate, event-driven notifications on job failures without polling or custom log parsing. It is the most operationally efficient method for alerting on Glue job failures, including those due to missing partitions.

Answer

Enable AWS Glue job bookmarks and configure an Amazon CloudWatch Logs subscription filter to email the engineer on error patterns.

Answer

Create an Amazon CloudWatch alarm on the Glue job's 'glue.driver.aggregate.numFailedTasks' metric and notify an Amazon SNS topic.

Answer

Schedule a daily AWS Lambda function that calls the Glue GetJobRuns API and sends an email if the last run failed.

Question 17

A data engineer is using AWS Lake Formation to manage access to a data lake in Amazon S3. The engineer needs to grant a specific IAM role read access to only the columns 'customer_id' and 'purchase_amount' in a table stored in the AWS Glue Data Catalog. The table contains sensitive columns like 'credit_card_number'. Which Lake Formation permission model should the engineer use to achieve this?

Accepted Answer

Grant the IAM role SELECT permission on the table, and then create a data filter that includes only the required columns.. AWS Lake Formation provides fine-grained access control, including column-level security. To grant read access to specific columns, you grant SELECT permission on the table and then attach a data filter that includes only the allowed columns. Data filters are evaluated at query time, ensuring that the role can only access the specified columns. This is the most direct and secure method.

Answer

Grant the IAM role DESCRIBE permission on the table, and then use AWS Glue to create a transformed dataset with only the required columns.

Answer

Grant the IAM role SELECT permission on the table, and then use an IAM policy to deny access to the sensitive columns.

Answer

Create a view in Amazon Athena that selects only the required columns, and grant the IAM role access to the view.

Question 18

A media company stores video metadata in an Amazon DynamoDB table. The security team requires that all data at rest in the table be encrypted with a customer managed key in AWS KMS, and that the key usage be auditable. The data engineer needs to configure encryption for the table. Which action should the data engineer take?

Accepted Answer

Create a customer managed KMS key and specify it when creating the DynamoDB table, ensuring the key policy allows DynamoDB to use it for encryption and decryption.. DynamoDB encryption at rest can use customer managed KMS keys. Specifying such a key during table creation ensures the table is encrypted with that key. The key policy must grant DynamoDB permission to use the key. KMS key usage is logged in CloudTrail, providing auditability. Other options use keys that are not customer managed or do not encrypt the table at rest.

Answer

Use DynamoDB Accelerator (DAX) with encryption in transit enabled, and configure the DAX cluster to use a customer managed KMS key.

Answer

Enable DynamoDB encryption at rest using the default AWS owned key, and enable CloudTrail logging for DynamoDB.

Answer

Enable DynamoDB encryption at rest using an AWS managed key, and use AWS CloudTrail to monitor key usage.

Question 19

A data engineer is using AWS Glue Studio to build a visual ETL job that reads JSON files from Amazon S3, applies a mapping transform, and writes Parquet to another S3 location. The engineer notices that the job is writing many small files, which hurts downstream query performance. Which action should the engineer take to reduce the number of output files without changing the source data?

Accepted Answer

Use the repartition or coalesce transform before the S3 target to reduce the number of output partitions.. The number of files written by a Glue job corresponds to the number of partitions in the DataFrame at write time. Repartition or coalesce reduces that partition count, so the job writes fewer, larger files. This directly addresses the small-file issue without altering the source data or the output format.

Answer

Increase the number of DPUs for the Glue job so more workers write files in parallel.

Answer

Enable the Glue job bookmark so the job only processes new files on each run.

Answer

Change the output format from Parquet to CSV so the files are smaller and easier to manage.

Question 20

A company's analytics team needs a petabyte-scale, fully managed data warehouse that supports standard SQL, columnar storage, and massively parallel query execution, and it must integrate with existing business intelligence tools with minimal operational effort. Which AWS service should the data engineer choose?

Accepted Answer

Amazon Redshift. The workload calls for a managed analytical database with columnar storage, massively parallel execution, and standard SQL connectivity for BI tools. Amazon Redshift delivers all three as a managed service, so the team avoids building and operating its own warehouse. The other services target transactional NoSQL access, in-memory caching, or data movement rather than petabyte-scale SQL analytics.

Answer

AWS Database Migration Service

Answer

Amazon ElastiCache for Redis

Answer

Amazon DynamoDB