DEA-C01 Data Operations and Support Practice Test 18 — 15 Questions

Question 1

A data engineer manages an AWS Glue ETL job that reads JSON files from Amazon S3, applies transformations, and writes Parquet files to another S3 location. The job has been running successfully for weeks. Recently, the job started failing with the error 'AnalysisException: Unable to infer schema for JSON. It must be specified manually.' The engineer confirms that the S3 path contains files but notices that the files are now compressed with gzip (.json.gz) whereas previously they were uncompressed. What is the MOST likely cause of the failure?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

AWS Glue cannot infer the schema from gzip-compressed JSON files when the file extension is .json.gz because it does not automatically decompress based on the extension.. The failure occurs because AWS Glue's JSON reader does not automatically detect gzip compression from the .json.gz file extension. When reading compressed JSON, you must explicitly set the compressionType parameter to 'gzip' in the Glue job's read options. Without this, Glue attempts to parse compressed bytes as plain JSON, leading to schema inference failure. The correct solution is to configure the reader with the appropriate compression type, not to change IAM permissions or assume corruption.

Answer

The JSON files are corrupted during the gzip compression process, causing Glue to fail schema inference.

Answer

The Glue job's IAM role lacks permissions to read objects with the .gz extension.

Answer

AWS Glue cannot read gzip-compressed JSON files because the compression codec is not supported.

Question 2

A data engineer is using Amazon Kinesis Data Streams to ingest real-time data. The stream has 4 shards. The engineer notices that the iterator age metric is increasing, indicating that consumers are falling behind. The consumer application is an AWS Lambda function with a batch size of 100 and a parallelization factor of 1. Which action will most effectively reduce iterator age?

Accepted Answer

Increase the Lambda function's parallelization factor to 10.. Increasing the parallelization factor allows multiple Lambda instances to process each shard concurrently, directly boosting consumer throughput. With 4 shards, a factor of 10 enables up to 40 concurrent invocations, which can significantly reduce iterator age. This is the most effective action because it scales processing without changing the stream's capacity or requiring code changes. Other options either do not address concurrency or are irrelevant to the consumer lag.

Answer

Increase the Lambda function's batch size to 500.

Answer

Increase the number of shards in the Kinesis stream to 8.

Answer

Enable enhanced fan-out for the Kinesis stream.

Question 3

A data engineer is using Amazon Kinesis Data Firehose to ingest streaming data into an Amazon S3 data lake. The data is in JSON format and is used for near-real-time analytics with Amazon Athena. The engineer notices that Athena queries are slow and return incorrect results because the JSON records have inconsistent schemas. The engineer wants to improve query performance and ensure data consistency without changing the ingestion mechanism. Which solution should the engineer implement?

Accepted Answer

Configure Kinesis Data Firehose to convert the JSON records to Apache Parquet using an AWS Glue Data Catalog schema.. Configuring Kinesis Data Firehose to convert JSON to Parquet using an AWS Glue Data Catalog schema enforces a consistent schema and columnar format, which improves Athena query performance and reduces cost. This is the most effective solution because it addresses both schema inconsistency and slow queries at the ingestion point.

Answer

Use AWS Glue crawlers to update the AWS Glue Data Catalog with the JSON schema and then query with Athena.

Answer

Increase the number of shards in the Kinesis Data Firehose delivery stream to improve throughput.

Answer

Enable Amazon S3 Select on the S3 bucket to allow Athena to push down predicates and reduce data scanned.

Question 4

A data engineer is configuring an AWS Glue ETL job that must read from an Amazon Redshift cluster and write curated results back to a different Redshift cluster. Connection tests succeed from the Glue console, but the job run fails when it attempts to read from the source table. The engineer needs to identify configuration requirements that must be satisfied for the job to read Redshift data successfully. (Choose two.)

Accepted Answer

The Glue connection must specify the Redshift cluster's JDBC URL, port, and a database user with SELECT privileges on the source table.. A Glue job reading Redshift data depends on a working JDBC path: correct endpoint, port, database, and credentials with SELECT privileges, plus network reachability from the job's VPC subnets and security groups to the cluster in an addressable Region. Console connection tests validate only basic reachability, so privileges and subnet or security group rules must be confirmed separately before the read succeeds.

Answer

The Glue job must be granted an IAM role that allows sts:AssumeRole on the Redshift service role used for COPY and UNLOAD operations.

Answer

The Redshift cluster must have the Glue Data Catalog enabled as an external schema before the job can issue SELECT statements.

Answer

The Glue job must enable job bookmarks on the Redshift source so that only new rows are read on each subsequent run.

Question 5

A data engineer runs an Apache Spark job on Amazon EMR that writes Parquet output to Amazon S3. The job uses dynamic partition overwrites and takes hours because it lists and deletes many small objects in the target prefix. The engineer needs to reduce job runtime while preserving the overwrite semantics. Which change should the engineer make?

Accepted Answer

Set spark.sql.sources.partitionOverwriteMode to dynamic and configure the EMRFS S3-optimized committer with the EMRFS fast upload and partition-level delete optimizations.. The EMRFS S3-optimized committer avoids expensive renames and, combined with dynamic partition overwrite, deletes at the partition level rather than enumerating every object. This directly addresses the hours spent listing and deleting small objects. The other options either keep the costly deletion path, mix incompatible committers, or assume the metastore deletes data files, which it does not.

Answer

Set spark.sql.sources.partitionOverwriteMode to dynamic and enable the Amazon S3 committer with partition-level delete via the EMRFS S3-optimized committer plus S3A magic committer.

Answer

Use the AWS Glue Data Catalog as the Spark metastore and enable Spark's dynamic partition overwrite mode, then rely on the catalog to remove stale partitions.

Answer

Enable the EMRFS S3-optimized committer by setting spark.sql.sources.partitionOverwriteMode to dynamic and using the EMRFS committer with the EMR File System (EMRFS) on Amazon EMR.

Question 6

A data engineer needs to schedule a daily AWS Glue ETL job that processes data from Amazon S3 and writes to Amazon Redshift. The job must run every day at 2:00 AM UTC. The engineer wants a simple, serverless scheduling solution with minimal operational overhead. Which AWS service should be used?

Accepted Answer

AWS Glue triggers with a schedule type of 'Scheduled' and a cron expression.. AWS Glue triggers are the native scheduling mechanism for Glue jobs. They support cron expressions and can start jobs on a schedule without additional services. This meets the requirement for a simple, serverless solution with minimal operational overhead. Other options either require extra services or are not designed for scheduling, making them less efficient for this scenario.

Answer

AWS Step Functions with a Wait state and a loop.

Answer

AWS Lambda with a cron expression in the function code.

Answer

Amazon EventBridge rule with a cron expression targeting the Glue job.

Question 7

A data engineer is using Amazon Kinesis Data Firehose to deliver streaming data to an Amazon S3 bucket. The delivery stream is configured with a 5-minute buffer interval and a 5 MB buffer size. The engineer notices that some data records are missing from S3. CloudWatch metrics show that the IncomingRecords metric is higher than the DeliveryToS3.Records metric. The engineer checks the S3 bucket and finds that the missing records are not present. What is the MOST likely reason for the missing records?

Accepted Answer

Kinesis Data Firehose is retrying failed records, and some records are being sent to the error output location because they failed to be delivered to S3 after retries.. The missing records are likely due to failed deliveries after retries. Kinesis Data Firehose attempts to deliver records to S3, but if a record fails (e.g., due to S3 permissions, throttling, or malformed data), it retries for a configured duration. If it still fails, the record is sent to the error output location (if configured) or discarded. The CloudWatch metrics show fewer records delivered than incoming, indicating such failures. The engineer should check the error output location and Firehose error logs to recover the missing data.

Answer

Kinesis Data Firehose is silently dropping records that exceed the 1 MB maximum record size.

Answer

Kinesis Data Firehose is delivering records to S3 but some records are being routed to a different destination due to a misconfigured dynamic partitioning.

Answer

The S3 bucket has versioning enabled, and the missing records were overwritten by newer deliveries.

Question 8

A data engineer is responsible for maintaining an Amazon Kinesis Data Stream that ingests clickstream data from a web application. The stream is configured with 10 shards. Recently, the application has experienced periods of high traffic, and the engineer notices that the WriteProvisionedThroughputExceeded metric is spiking. The engineer needs to ensure the stream can handle the increased load without manual intervention. Which TWO actions should the engineer take? (Choose two.)

Accepted Answer

Switch the stream to on-demand capacity mode.. The WriteProvisionedThroughputExceeded metric indicates that the stream's write capacity is insufficient for the current load. Increasing the number of shards manually or switching to on-demand mode both increase write capacity and allow the stream to handle higher traffic without throttling. These actions directly address the root cause and can be implemented without downtime.

Answer

Enable enhanced fan-out for consumers to increase read throughput.

Answer

Use the Kinesis Producer Library (KPL) to aggregate records before sending.

Answer

Increase the retention period of the stream to 168 hours.

Question 9

A data engineer is responsible for an Amazon Redshift cluster that runs nightly ETL jobs. The engineer needs to ensure that the cluster has enough disk space for the ETL load and that queries perform well. The cluster currently has a single RA3 node type. The engineer observes that during the ETL load, the cluster's disk usage reaches 90% and query performance degrades. Which action should the engineer take to address the disk space issue with the least operational overhead?

Accepted Answer

Resize the cluster to add more nodes of the same RA3 node type.. RA3 nodes decouple compute and storage, allowing storage to scale independently by adding nodes. Adding nodes increases the cluster's managed storage capacity, providing the necessary disk space for the ETL load. This approach has low operational overhead because it can be done with elastic resize, and it also adds compute resources that can improve query performance.

Answer

Increase the cluster's snapshot retention period to free up disk space.

Answer

Enable concurrency scaling to offload read queries to additional clusters.

Answer

Change the cluster to use dense storage nodes (DS2) to increase local disk capacity.

Question 10

A data engineer runs an AWS Glue crawler over an Amazon S3 prefix that contains JSON files whose schema evolves: older files lack a new `campaign_id` field that newer files include. After each crawl, the AWS Glue Data Catalog table schema changes and Athena queries intermittently fail with schema mismatch errors. The engineer wants Athena to read all files and return NULL for `campaign_id` in the older records. Which configuration should be applied to the crawler?

Accepted Answer

Configure the crawler to combine schemas across all files by setting the schema change policy to update the table and using a JSON classifier that treats each file as a record.. Crawlers discover schemas per file and can reconcile them into a single table definition when configured to update the catalog. Merging the discovered columns yields a unified schema containing `campaign_id`. Because Athena applies schema-on-read to JSON, files that omit that key return NULL for the column rather than failing, which satisfies the requirement without rewriting the source data.

Answer

Configure the crawler with the 'Ignore the change and don't update the table in the Data Catalog' schema change policy.

Answer

Convert the JSON files to Parquet using an AWS Glue ETL job before crawling so the crawler sees a single consistent schema.

Answer

Set the crawler's schema change policy to Update the table definition in the Data Catalog and use a classifier that merges schemas.

Question 11

A data engineer is managing an Amazon Redshift cluster that experiences performance degradation during complex analytical queries. The engineer needs to implement strategies to improve query performance. Which TWO actions should be taken? (Choose two.)

Accepted Answer

Enable Concurrency Scaling to handle bursts of concurrent queries.. Concurrency Scaling and workload management (WLM) are both effective strategies to improve query performance in Amazon Redshift. Concurrency Scaling handles bursts of concurrent queries by adding transient capacity, while WLM prioritizes queries to reduce contention. Together, they address performance degradation from concurrency and resource allocation. Other options like adding nodes or disabling caching do not directly target the described issue and may even be harmful.

Answer

Disable Result Caching to ensure fresh data for every query.

Answer

Implement automatic vacuum and analyze operations.

Answer

Increase the number of slices by adding more nodes to the cluster.

Question 12

A data engineer is using AWS Step Functions to orchestrate a series of AWS Glue jobs. The state machine passes the output of one Glue job as input to the next. The engineer notices that the second Glue job is failing with a 'ResourceNotFoundException' because it cannot find the input data. The first Glue job writes its output to an Amazon S3 location, and the second job reads from that location. The engineer verifies that the first job completes successfully and writes data to S3. What is the MOST likely cause of the failure?

Accepted Answer

The first Glue job writes its output to S3, but the second Glue job is configured to read from a different S3 path than the one the first job writes to.. The ResourceNotFoundException indicates that the second Glue job cannot find the data at the specified S3 location. Since the first job completes successfully and writes data, the most likely cause is a mismatch between the S3 path written by the first job and the S3 path read by the second job. This could be due to a typo, a different bucket, or a different prefix. The engineer should inspect both job scripts and the Step Functions state machine to ensure the paths align.

Answer

The Step Functions state machine is passing the wrong output from the first Glue job to the second, causing the second job to look in the wrong S3 location.

Answer

The second Glue job's IAM role lacks permissions to read from the S3 bucket where the first job wrote its output.

Answer

The Step Functions state machine does not have the necessary IAM permissions to read the output of the first Glue job.

Question 13

A data engineer is using AWS Lake Formation to manage fine-grained access control on a data lake stored in Amazon S3. The engineer grants a user SELECT permission on a table in the AWS Glue Data Catalog using Lake Formation. However, when the user runs an Amazon Athena query on that table, they receive an 'Access Denied' error. The user has the necessary IAM permissions to run Athena queries and access the S3 bucket. What is the most likely cause of the error?

Accepted Answer

The user's IAM role does not have the lakeformation:GetDataAccess permission.. Lake Formation uses a credential vending mechanism to provide temporary access to data in S3. The query engine's IAM role must have lakeformation:GetDataAccess to obtain these credentials. Without this permission, Lake Formation denies the data access request, causing an 'Access Denied' error in Athena even though the user has SELECT permission on the table.

Answer

The AWS Glue Data Catalog table is not registered with Lake Formation.

Answer

The S3 bucket policy does not allow the user's IAM role to access the data.

Answer

The user's IAM role lacks the athena:StartQueryExecution permission.

Question 14

A data engineer is monitoring an AWS Glue ETL job that reads from an Amazon S3 bucket and writes to Amazon Redshift. The job has been running successfully but recently started taking much longer than usual. The engineer suspects that the job is not efficiently utilizing resources and wants to optimize it. Which TWO actions should the engineer take to improve the job's performance? (Choose two.)

Accepted Answer

Enable AWS Glue job metrics to identify bottlenecks in the job execution.. To optimize a slow AWS Glue job, first enable job metrics to identify bottlenecks, then consider increasing DPUs if resource constraints are evident. These actions address both diagnosis and remediation. Other options like changing output format or disabling bookmarks would likely harm performance or increase cost.

Answer

Use AWS Glue's automatic scaling to dynamically adjust resources based on workload.

Answer

Convert the output format to CSV to reduce write time to Amazon Redshift.

Answer

Disable job bookmarks to allow reprocessing of all data on each run.

Question 15

A company uses Amazon Kinesis Data Streams to ingest clickstream data. The data is consumed by a custom consumer application that writes to Amazon S3 every 5 minutes. The consumer is falling behind and processing lag is increasing. Which action is MOST effective to reduce the lag?

Accepted Answer

Increase the number of shards in the Kinesis stream. The consumer is falling behind because the stream's throughput capacity is insufficient for the incoming data volume. Increasing the number of shards in the Kinesis stream directly increases the total read capacity (each shard provides 2 MB/s read throughput and 5 transactions/second), allowing the consumer to process more data in parallel and reduce lag.

Answer

Switch to Amazon Kinesis Data Firehose to deliver data directly to S3

Answer

Increase the batch size of records written to S3

Answer

Reduce the retention period of the stream