DEA-C01 Data Operations and Support Practice Test 2 — 15 Questions

Question 1

A data engineer is troubleshooting an AWS Glue ETL job that fails with the error 'java.lang.OutOfMemoryError: Java heap space'. The job processes a large number of small files in Amazon S3. Which action would MOST effectively resolve the issue?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Enable S3 groupFiles option in the Glue job. Enabling the S3 groupFiles option in AWS Glue groups small files into fewer, larger splits, which reduces the number of Spark partitions and memory overhead, directly addressing the 'Java heap space' error. Option B is wrong because changing the worker type to G.1X (which has more memory per worker) does not solve the root cause of too many small files; it only provides more memory per worker but still processes many small files. Option C is wrong because increasing the number of workers increases parallelism but does not reduce the memory overhead per worker caused by numerous small files. Option D is wrong because using a G.2X worker type with more memory may temporarily alleviate the issue but is less effective than grouping files, which reduces the number of splits and thus memory consumption at the source.

Answer

Change the worker type to G.1X

Answer

Increase the number of workers in the Glue job

Answer

Use a G.2X worker type with more memory

Question 2

A company uses Amazon EMR to run Spark jobs on data stored in S3. After upgrading the EMR cluster to a new release, one of the Spark jobs fails with 'OutOfMemoryError' in the executor. Which configuration change is MOST likely to resolve this issue?

Accepted Answer

Increase spark.executor.memory to allocate more memory per executor.. Increasing spark.executor.memory directly allocates more memory per executor, addressing the OutOfMemoryError in the executor. Option A (increasing the number of core nodes) adds more cluster capacity but does not increase the memory available to individual executors, so it may not resolve the OOM if the existing executors are already memory-constrained. Option B (decreasing spark.sql.shuffle.partitions) reduces the number of shuffle partitions, which can increase the size of each partition and potentially cause more memory pressure, not less. Option C (increasing spark.driver.memory) only helps the driver process, not the executor, so it does not fix executor OOM errors.

Answer

Increase the number of core nodes in the EMR cluster.

Answer

Decrease spark.sql.shuffle.partitions to reduce overhead.

Answer

Increase spark.driver.memory in the Spark configuration.

Question 3

A company uses Amazon Redshift for its data warehouse. During a routine audit, the data engineer discovers that some queries are returning stale data even though the underlying source data has been updated. The engineer confirms that the COPY command completes successfully and that no errors are reported. Which action should the engineer take to ensure queries reflect the latest data?

Accepted Answer

Refresh the materialized views that the queries are using.. Materialized views in Amazon Redshift are not automatically refreshed when the underlying base tables are updated via COPY. To ensure queries return the latest data, the materialized views must be manually refreshed using the REFRESH MATERIALIZED VIEW command. Option A (VACUUM) reclaims disk space and re-sorts rows but does not update materialized views. Option B (RESET ALL) clears the query result cache, but stale materialized views persist even if the cache is cleared. Option C (ANALYZE) updates table statistics for query planning, not the actual data content. Therefore, refreshing the materialized views is the necessary action.

Answer

Run the VACUUM command on the source tables.

Answer

Clear the Redshift result cache by running RESET ALL.

Answer

Run the ANALYZE command on the source tables.

Question 4

A data engineer configured an AWS Glue job that reads from an S3 bucket and writes to an Amazon Redshift table. The job runs successfully, but the data in Redshift is missing some records that exist in S3. The engineer suspects the issue may be related to the job's bookmarks. What should the engineer do to ensure all records are processed?

Accepted Answer

Disable job bookmarks in the Glue job configuration.. Disabling job bookmarks forces the Glue job to reprocess all data from the S3 source, ensuring no records are missed. Option A is incorrect because increasing the number of workers does not address bookmark-related issues. Option B is incorrect because resetting the job bookmark using the AWS Glue API is not a standard operation; bookmarks are managed via the job configuration. Option C is incorrect because updating the IAM role does not affect bookmark behavior.

Answer

Update the IAM role to grant additional S3 read permissions.

Answer

Reset the job bookmark using the AWS Glue API.

Answer

Increase the number of workers in the Glue job.

Question 5

A data engineer notices that an AWS Glue ETL job is failing intermittently with the error 'Connection refused'. The job reads from Amazon RDS for MySQL and writes to Amazon S3. What is the MOST likely cause?

Accepted Answer

The security group for the RDS instance is not allowing inbound traffic from the Glue job's subnet.. The 'Connection refused' error indicates a network connectivity issue between the AWS Glue ETL job and Amazon RDS for MySQL. The most likely cause is that the security group for the RDS instance is not configured to allow inbound traffic on the MySQL port (3306) from the subnet or security group used by the Glue job. Option A (max connections) would result in a 'too many connections' error, not 'connection refused'. Option C (DPU limits) would cause resource-related errors, not a connection refusal. Option D (IAM permissions) would lead to an access denied error when writing to S3, not a connection error. Therefore, option B is correct.

Answer

The RDS instance has reached its maximum number of connections.

Answer

The Glue job is using too many DPUs and hitting resource limits.

Answer

The IAM role associated with the Glue job lacks permissions to write to the S3 bucket.

Question 6

A data engineer is setting up an Amazon Redshift cluster for a new data warehouse. The engineer needs to ensure that the cluster can automatically recover from failures and maintain high availability. Which TWO actions should the engineer take? (Choose TWO.)

Accepted Answer

Configure automated snapshots with a retention period of 1 day and enable cluster recreation from snapshots.. (Configure automated snapshots with a retention period of 1 day and enable cluster recreation from snapshots) and Option C (Enable Multi-AZ deployment) are correct. Amazon Redshift supports Multi-AZ deployments for high availability, providing automatic failover to a standby cluster in another Availability Zone in case of failure. Automated snapshots allow point-in-time recovery and enable the cluster to be recreated from a snapshot if the primary cluster fails, ensuring data durability and recovery. Option A is incorrect because WLM queues manage query concurrency and priority, not availability. Option D is incorrect because concurrency scaling improves performance for concurrent queries, not availability. Option E is incorrect because manual snapshots require manual intervention and do not automatically recover from failures.

Answer

Configure workload management (WLM) queues to prioritize critical queries.

Answer

Enable concurrency scaling.

Answer

Configure manual snapshots with cross-region copy.

Question 7

A company is using AWS Glue DataBrew to clean and transform data from an S3 bucket. The data contains personally identifiable information (PII). The company wants to mask the PII columns before making the dataset available to analysts. Which THREE actions can the engineer perform using DataBrew to mask PII? (Choose THREE.)

Accepted Answer

Apply a 'Column masking' transformation to replace values with 'XXX'.. Options A, C, and D are correct. DataBrew provides built-in transformations for masking: 'Column masking' replaces values with a fixed pattern, 'Column hashing' replaces with a hash, and 'Column delete' removes the column. Option B is wrong because 'Column tokenization' is not a built-in DataBrew transformation; tokenization would require a custom recipe step. Option E is wrong because 'Column encryption' is not a DataBrew transformation; encryption is typically handled at rest or with AWS KMS, not within DataBrew.

Answer

Apply a 'Column tokenization' transformation to replace values with tokens.

Answer

Apply a 'Column encryption' transformation to encrypt the column values.

Question 8

A data engineer reviews the above error log from an AWS Glue ETL job. The job uses a G.1X worker type (16 GB memory). The job processes a 30 GB CSV file from S3. What should the engineer do to resolve the memory error?

Accepted Answer

Set 'spark.executor.memory' to 12g in the job parameters.. The error indicates that the executor ran out of memory (10 GB used of 10 GB limit). Increasing the Spark executor memory to 12 GB (since G.1X has 16 GB total, leaving room for overhead) will prevent the container from being killed. Option A is wrong because converting CSV to Parquet reduces data size but does not increase the per-executor memory limit. Option C is wrong because decreasing the number of workers reduces parallelism and may increase memory pressure per executor. Option D is wrong because increasing the number of workers adds more executors but does not increase per-executor memory.

Answer

Convert the input file from CSV to Parquet format.

Answer

Decrease the number of workers to 1 to reduce memory overhead.

Answer

Increase the number of workers from 2 to 4.

Question 9

A data engineering team uses Amazon EMR to run Spark jobs on a transient cluster. The jobs read data from S3 and write results back to S3. The team notices that jobs are taking longer than expected. Which configuration change is most likely to improve performance?

Accepted Answer

Enable EMRFS consistent view and use S3 as the direct input source.. Enabling EMRFS consistent view ensures that Spark jobs see the latest data from S3, avoiding retries due to eventual consistency and improving performance. Option A is incorrect because the master node manages the cluster but does not process data; a larger instance type for the master node does not improve job execution speed. Option B is incorrect because using HDFS as intermediate storage adds extra data copy steps and increases I/O overhead, which can slow down jobs. Option C is incorrect because while increasing core nodes improves parallelism, the primary performance issue in this scenario is likely due to S3 consistency rather than parallelism; thus, enabling EMRFS consistent view addresses the root cause more effectively.

Answer

Use a larger instance type for the master node.

Answer

Enable HDFS as the intermediate storage and copy data from S3 to HDFS before processing.

Answer

Increase the number of core nodes to improve parallelism.

Question 10

Your team uses Amazon Kinesis Data Analytics to process real-time streaming data from an Amazon Kinesis Data Stream. The application calculates windowed aggregations and writes results to an Amazon S3 bucket using a delivery stream. Recently, the application has been failing with a 'LimitExceededException' when writing to the delivery stream. You have checked the CloudWatch metrics and see that the IncomingBytes and IncomingRecords for the delivery stream are well below the provisioned limits. The delivery stream has a buffer size of 5 MB and a buffer interval of 60 seconds. The application generates about 500 records per second, each about 1 KB. What is the most likely cause and correct action?

Accepted Answer

Increase the buffer size of the delivery stream to 10 MB to accommodate larger writes.. The LimitExceededException in Kinesis Data Firehose typically occurs when the rate of API calls exceeds the transaction limit (5,000 calls per second per delivery stream). With 500 records per second, using PutRecord would exceed this limit because each call sends a single record. Even with PutRecordBatch, if batch sizes are small, the call rate remains high. Increasing the buffer size to 10 MB allows the application to batch more records per call, reducing the number of calls and staying under the transaction limit. Option B is incorrect because reducing batch sizes increases the call rate, worsening the problem.

Answer

Increase the number of shards in the Kinesis Data Stream to reduce the load on the application.

Answer

Modify the application to use PutRecordBatch with smaller batch sizes to stay within the 4 MB per-call limit.

Answer

Reduce the buffer interval of the delivery stream to 30 seconds to flush data more frequently.

Question 11

Your company has an Amazon S3-based data lake partitioned by year/month/day. An AWS Glue crawler runs daily to update the Data Catalog. A Spark job on Amazon EMR reads the latest partition and performs transformations. Recently, the Spark job has been failing with a 'FileNotFoundException' for a file that is expected to exist. You check the S3 bucket and see that the file exists. The job is configured to use S3 as the direct input source with EMRFS consistent view enabled. The IAM role for the EMR cluster has full S3 access. What is the most likely cause?

Accepted Answer

The EMR cluster is experiencing eventual consistency issues despite EMRFS consistent view.. While EMRFS consistent view improves consistency, it does not guarantee immediate visibility of objects written by external systems like the Glue crawler. This can cause a FileNotFoundException when the Spark job attempts to read a recently written file. Options A and D are not supported, and option B is incorrect because permissions are sufficient.

Answer

The Data Catalog partition metadata is out of sync because the crawler runs after the job.

Answer

The Glue crawler did not update the Data Catalog because of insufficient permissions.

Answer

The file was deleted by a lifecycle policy after the job started.

Question 12

A data engineering team uses AWS Glue ETL jobs to process daily data from an Amazon RDS for PostgreSQL instance into Amazon S3. Recently, the jobs have been failing randomly with the error 'psycopg2.OperationalError: could not connect to server: Connection timed out'. The RDS instance is in a private subnet with a security group that allows inbound traffic from the Glue job's security group on port 5432. The Glue job is configured to use the same VPC, subnet, and security group. The RDS instance has sufficient connections and is not at CPU or memory limits. The failures occur at different times each day, and the job works when retried immediately. Which action should the team take to resolve the issue?

Accepted Answer

Increase the number of retries in the Glue job configuration to 5.. The error 'Connection timed out' indicates intermittent network connectivity issues, not IP exhaustion. Increasing retries allows the job to succeed on retry after transient network failures resolve. Option B is incorrect because subnet IP exhaustion would cause an ENI allocation error, not a timeout. Option A is incorrect because timeout settings do not address connectivity. Option C is incorrect because the existing security group rule on port 5432 is sufficient; ephemeral ports are not needed for PostgreSQL.

Answer

Set the Glue job timeout to 60 minutes to ensure the job does not fail prematurely.

Answer

Change the subnet of the Glue job to one with a larger CIDR range (e.g., /20 instead of /24).

Answer

Add an inbound rule to the RDS security group allowing traffic on ports 1024-65535 from the Glue security group.

Question 13

A data engineer notices that an AWS Glue job processing data from an Amazon S3 bucket frequently fails with 'OutOfMemoryError'. The job reads CSV files, applies transformations, and writes Parquet to another S3 bucket. The job has 10 workers of type G.1X. Which change is MOST likely to resolve the issue?

Accepted Answer

Change the worker type from G.1X to G.2X. The G.1X worker type provides 16 GB of memory per worker. An OutOfMemoryError indicates that the job's memory requirements exceed this limit. Upgrading to G.2X doubles the memory per worker to 32 GB, directly addressing the memory shortage without changing the parallelism or incurring the overhead of additional workers.

Answer

Increase the number of workers to 20

Answer

Change the worker type from G.1X to G.8X

Answer

Enable the Spark UI to monitor memory and tune the job

Question 14

A company uses Amazon Kinesis Data Streams to ingest clickstream data. The data is consumed by a custom consumer application that writes to Amazon S3 every 5 minutes. The consumer is falling behind and processing lag is increasing. Which action is MOST effective to reduce the lag?

Accepted Answer

Increase the number of shards in the Kinesis stream. The consumer is falling behind because the stream's throughput capacity is insufficient for the incoming data volume. Increasing the number of shards in the Kinesis stream directly increases the total read capacity (each shard provides 2 MB/s read throughput and 5 transactions/second), allowing the consumer to process more data in parallel and reduce lag.

Answer

Switch to Amazon Kinesis Data Firehose to deliver data directly to S3

Answer

Increase the batch size of records written to S3

Answer

Reduce the retention period of the stream

Question 15

A data team runs a daily AWS Glue ETL job that processes data from an Amazon Redshift cluster and writes results to Amazon S3. The job completes successfully but takes 2 hours longer than expected. The job uses the JDBC connection to Redshift. The Redshift cluster is 4 dc2.large nodes. The Glue job has 10 workers of type G.1X. Which change would MOST likely reduce the job duration?

Accepted Answer

Use the S3 staging option in the Glue connection to unload data from Redshift to S3 first. The JDBC connection in AWS Glue reads data row-by-row from Redshift, which is slow for large datasets. By enabling the S3 staging option in the Glue connection, the job uses Redshift's UNLOAD command to export data to S3 in parallel, then Glue reads from S3. This bypasses the JDBC bottleneck and leverages Redshift's massively parallel processing (MPP) to export data much faster.

Answer

Use Redshift Spectrum to query data directly from S3

Answer

Increase the Redshift cluster size to 8 nodes

Answer

Increase the number of Glue workers to 20