DEA-C01 Data Operations and Support Practice Test 3 — 15 Questions

Question 1

A company uses Amazon DynamoDB as a source for an AWS Glue job. The job reads a large table using a DynamoDB export to S3 feature. The job is failing with 'ThrottlingException' from DynamoDB. What should the data engineer do to resolve this issue WITHOUT changing the job's logic?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use the DynamoDB export to S3 feature and read the exported data from S3. The DynamoDB export to S3 feature creates a point-in-time snapshot of the table data in S3 without consuming any read capacity units (RCUs) from the DynamoDB table. By reading the exported data from S3 instead of directly scanning the DynamoDB table, the Glue job avoids triggering ThrottlingException entirely, as the export operation uses the table's backup and restore mechanism, not the read path. This resolves the issue without altering the job's logic, as the job can be reconfigured to read from the S3 export location.

Answer

Use DynamoDB Streams to capture changes and process them incrementally

Answer

Reduce the number of DynamoDB read segments in the Glue job

Answer

Increase the read capacity units (RCU) of the DynamoDB table

Question 2

A data engineer is monitoring an Amazon Kinesis Data Analytics application that uses a SQL query to aggregate streaming data. The application is falling behind and the millisBehindLatest metric is increasing. Which action should the engineer take to improve performance?

Accepted Answer

Increase the Parallelism setting of the Kinesis Data Analytics application. Increasing the Parallelism setting of the Kinesis Data Analytics application allows the SQL query to process data across more in-application streams and operators concurrently, directly addressing the lag indicated by the rising millisBehindLatest metric. This action scales the compute resources allocated to the application without changing the source stream or the query logic, making it the most direct way to improve throughput for a SQL-based Kinesis Data Analytics application.

Answer

Switch from SQL to Apache Flink for the analytics application

Answer

Increase the number of shards in the source Kinesis stream

Answer

Decrease the window duration of the SQL query

Question 3

A data engineer is troubleshooting an AWS Glue job that reads from an Amazon RDS for PostgreSQL database using a JDBC connection. The job fails with the error 'java.sql.SQLException: No suitable driver'. Which TWO actions should the engineer take to resolve this issue? (Select TWO.)

Accepted Answer

Verify that the connection string in the job's JDBC URL uses the correct format and includes the driver class. The 'No suitable driver' error in JDBC indicates that the driver class specified in the JDBC URL is either missing or incorrect. For PostgreSQL, the JDBC URL must follow the format 'jdbc:postgresql://host:port/database' and the driver class must be 'org.postgresql.Driver'. If the URL is malformed or the driver class is not properly referenced, the Glue job cannot load the driver, leading to this specific SQLException.

Answer

Check that the Glue job's VPC and security groups allow outbound traffic to the RDS instance

Answer

Restart the Glue job with a higher timeout value

Answer

Update the IAM role associated with the Glue job to allow 'rds:*' permissions

Question 4

A company uses Amazon S3 to store raw data and runs AWS Glue ETL jobs to transform it into Parquet. The data is then queried using Amazon Athena. Queries are slow and expensive due to high scan volumes. Which THREE design changes can improve query performance and reduce costs? (Select THREE.)

Accepted Answer

Convert the data to a columnar format like Parquet or ORC if not already. Columnar formats like Parquet or ORC store data by column rather than by row, allowing Athena to read only the columns needed for a query. This drastically reduces the amount of data scanned per query, directly lowering both latency and cost since Athena charges based on the volume of data read.

Answer

Increase the number of files by reducing file size to 1 MB

Answer

Use bucketing on high-cardinality columns

Question 5

A data engineer runs a Spark job on Amazon EMR that reads data from Amazon S3 and writes results back to S3. The job fails with an 'S3AccessDenied' error. The engineer verifies that the IAM role attached to the EMR cluster has s3:GetObject and s3:PutObject permissions on the relevant buckets. What is the MOST likely cause of the error?

Accepted Answer

The IAM role does not have s3:ListBucket permission on the bucket.. The IAM role attached to the EMR cluster must have the s3:ListBucket permission on the bucket to allow the Spark job to enumerate objects when reading from S3. Without this permission, even with s3:GetObject and s3:PutObject, the job fails with an 'S3AccessDenied' error because the S3 list operation is required for directory listing and file discovery.

Answer

S3 Transfer Acceleration is not enabled on the bucket.

Answer

EMRFS consistent view is not configured.

Answer

The S3 bucket is in a different AWS Region than the EMR cluster.

Question 6

An AWS Glue job that processes streaming data from Amazon Kinesis Data Streams is failing intermittently with 'Failed to checkpoint' errors. The job uses checkpointing to an Amazon S3 bucket every 60 seconds. Which action should the engineer take to resolve the issue?

Accepted Answer

Increase the checkpoint interval to 120 seconds.. The 'Failed to checkpoint' error in AWS Glue streaming jobs typically occurs when the checkpoint operation exceeds the 60-second interval due to high throughput or large state size. Increasing the checkpoint interval to 120 seconds provides more time for the checkpoint to complete, reducing the likelihood of timeouts and allowing the job to stabilize without losing progress.

Answer

Move the checkpoint location to an Amazon DynamoDB table.

Answer

Decrease the Kinesis shard count to reduce throughput.

Answer

Disable checkpointing and rely on Kinesis iterator age.

Question 7

A company uses AWS DMS to migrate data from an on-premises Oracle database to Amazon Redshift. The migration is successful, but after a few days, data in Redshift becomes inconsistent with the source due to ongoing changes. The company needs to keep Redshift synchronized with minimal latency. Which approach should the data engineer use?

Accepted Answer

Configure DMS with ongoing replication using change data capture (CDC).. AWS DMS supports ongoing replication using change data capture (CDC), which captures incremental changes from the Oracle source (via Oracle LogMiner or binary logs) and applies them to Amazon Redshift in near real-time. This approach ensures that Redshift remains synchronized with the source database with minimal latency, meeting the requirement for ongoing consistency after the initial full load.

Answer

Use Amazon Redshift COPY with S3 staging and AWS Lambda triggers.

Answer

Schedule a full DMS load every night.

Answer

Set up Amazon Redshift Spectrum to query the Oracle database directly.

Question 8

A data engineer notices that an Amazon Kinesis Data Firehose delivery stream is failing to deliver data to an Amazon S3 bucket. The CloudWatch metrics show 'DeliveryToS3.Success' is 0 and 'S3.BucketExists' is 1. What is the MOST likely cause?

Accepted Answer

The IAM role for Firehose lacks s3:PutObject permission.. The metric 'S3.BucketExists' is 1, confirming the S3 bucket exists, so the issue is not bucket existence. With 'DeliveryToS3.Success' at 0, the failure is in the write operation. The IAM role assumed by Firehose must have the s3:PutObject permission to deliver data; lacking it would cause all delivery attempts to fail silently, matching the observed metrics.

Answer

The S3 bucket has an ACL that denies access to Firehose.

Answer

The Firehose delivery stream Lambda transformation function is failing.

Answer

The S3 bucket does not exist.

Question 9

A company runs a batch ETL job on Amazon EMR every night. Recently, the job started failing with 'Out of Memory' errors in the Spark executors. The data volume has grown 20% in the past month. The cluster uses uniform instance groups with 5 core nodes of r5.xlarge (4 vCPU, 32 GB RAM). Which change should the data engineer implement to resolve the issue with minimal cost increase?

Accepted Answer

Configure instance fleets to include r5.xlarge and r5.2xlarge instances.. Using instance fleets allows the cluster to include both r5.xlarge and r5.2xlarge instances, enabling the Spark executors to use the larger instances for memory-intensive tasks while still leveraging the existing r5.xlarge nodes. This provides a cost-effective way to handle the 20% data growth by adding memory capacity without replacing the entire cluster or over-provisioning all nodes. Instance fleets also support Spot Instances, which can further reduce costs while addressing the Out of Memory errors.

Answer

Increase the number of core nodes to 7.

Answer

Change instance type to r5.2xlarge (8 vCPU, 64 GB RAM) for all nodes.

Answer

Tune Spark memory configurations to reduce executor memory overhead.

Question 10

A data engineer is troubleshooting an AWS Glue ETL job that fails with the error: 'An error occurred while calling o123.pyWriteDynamicFrame. Access Denied when writing to S3 bucket: my-bucket'. The job uses a Glue service role named 'GlueServiceRole'. Which TWO actions should the engineer take to resolve the issue? (Choose TWO.)

Accepted Answer

Check if the S3 bucket policy denies access from the GlueServiceRole.. The error message indicates an access denied when writing to S3, which can be caused by a bucket policy that explicitly denies the Glue service role's access, even if the IAM policy allows it. Option D is correct because the IAM policy attached to GlueServiceRole must include the s3:PutObject permission on the specific bucket to allow the Glue job to write data.

Answer

Disable S3 Block Public Access on the bucket.

Answer

Grant the GlueServiceRole permission to write to the AWS Glue Data Catalog.

Answer

Ensure the Glue job is in the same VPC as the S3 bucket.

Question 11

A data engineer is monitoring an Amazon Kinesis Data Analytics for Apache Flink application that processes streaming data. The application is falling behind (increasing 'MillisBehindLatest') and the CPU utilization of the Flink task managers is consistently above 80%. Which THREE actions should the engineer take to improve performance? (Choose THREE.)

Accepted Answer

Increase the number of shards in the Kinesis data stream.. Increasing the number of shards in the Kinesis data stream (Option A) directly increases the ingestion capacity and parallelism source for the Flink application. With more shards, the application can read data from more partitions concurrently, reducing the backlog indicated by 'MillisBehindLatest'. This is a fundamental scaling action for Kinesis-based Flink applications.

Answer

Decrease the checkpoint interval to reduce state size.

Answer

Decrease the number of task managers to reduce CPU contention.

Question 12

A data engineer is troubleshooting a nightly AWS Glue ETL job that reads from an Amazon RDS for MySQL table and writes to an Amazon S3 bucket in Parquet format. The job runs successfully most days, but occasionally fails with the error 'ERROR: An error occurred while calling o67.pyWriteDynamicFrame. The transaction log for the database is full due to 'LOG_BACKUP'.' What is the MOST likely cause of this error?

Accepted Answer

The MySQL database transaction log needs to be backed up to free space.. The error message 'The transaction log for the database is full due to 'LOG_BACKUP'' indicates that the MySQL database's transaction log has reached its maximum size because it has not been backed up and truncated. In MySQL, the transaction log (often the InnoDB redo log or binary log) must be backed up periodically to free space; otherwise, write operations fail. This is a database-side issue, not a Glue or permissions problem, so the correct action is to back up the transaction log to release space.

Answer

The MySQL database has reached its maximum number of concurrent connections.

Answer

The AWS Glue job does not have sufficient permissions to write to the S3 bucket.

Answer

The AWS Glue job is configured with an incorrect 'writeDynamicFrame' method.

Question 13

A company runs a data pipeline that ingests clickstream data from a web application into Amazon Kinesis Data Streams. A Lambda function processes records from the stream and writes them to an Amazon S3 bucket in JSON format. The pipeline has been running smoothly, but for the past hour, the Lambda function has been failing with 'Rate exceeded' errors, and the Kinesis stream shows elevated 'IteratorAgeMilliseconds' metrics. The Lambda function has a reserved concurrency of 100, and the Kinesis stream has 10 shards. The average record size is 5 KB, and the data rate is approximately 15 MB per second. Which combination of actions should a data engineer take to resolve the issue and prevent recurrence? (Choose TWO.)

Accepted Answer

Increase the Lambda function's reserved concurrency to 200.. The 'Rate exceeded' errors indicate that the Lambda function's concurrency is insufficient to keep up with the incoming data rate from Kinesis. With 10 shards and a 15 MB/s data rate, each shard processes ~1.5 MB/s, and with 5 KB records, that's ~300 records per second per shard. Increasing reserved concurrency to 200 allows more parallel invocations to handle the load, reducing the iterator age.

Answer

Decrease the Lambda function's batch size from 100 to 50.

Answer

Enable S3 multipart upload for the Lambda function.

Answer

Replace the Lambda function with Amazon Kinesis Data Firehose to write directly to S3.

Question 14

A company uses AWS Glue to run ETL jobs that process data from Amazon S3 and write results to Amazon Redshift. The Glue job uses the JDBC connection to Redshift. Recently, the job has been failing intermittently with the error: 'java.sql.SQLException: [Amazon](500310) Invalid operation: INSERT has more expressions than target columns;' The Glue job writes to a staging table in Redshift before performing a merge into the final table. The staging table schema matches the source data. The error occurs only on some days and affects different columns each time. The data engineer suspects that the source data occasionally contains extra columns due to a schema drift in the upstream data producer. Which approach should the data engineer take to handle this issue robustly?

Accepted Answer

Use a Glue DynamicFrame and apply the resolveChoice method to make the schema consistent.. Glue DynamicFrames can automatically handle schema drift using the `resolveChoice` method, which allows you to specify how to handle columns that appear inconsistently across records (e.g., making them null, casting to a common type, or dropping them). This directly addresses the intermittent error caused by extra columns in the source data without requiring manual schema updates or fragile conditional logic.

Answer

Skip any records that have extra columns by adding a conditional check in the Glue script.

Answer

Manually update the Redshift staging table schema whenever the source data changes.

Answer

Use a Glue DynamicFrame and apply the dropFields method to remove extra columns before writing.

Question 15

A data engineer is troubleshooting a Glue ETL job that reads from an S3 bucket and writes to a Redshift table. The job fails with a 'MemoryError' when processing a large dataset. Which TWO actions should the engineer take to resolve this issue? (Choose TWO.)

Accepted Answer

Increase the number of DPUs and set 'spark.sql.shuffle.partitions' to a higher value.. Increasing the number of DPUs (Data Processing Units) provides more memory and compute resources to the Glue job, directly addressing the MemoryError. Setting 'spark.sql.shuffle.partitions' to a higher value reduces the amount of data shuffled per partition, preventing out-of-memory errors during wide transformations like joins or aggregations.

Answer

Increase the number of DPUs and set 'coalesce(1)' in the script.

Answer

Decrease the number of DPUs and increase 'spark.shuffle.partitions'.

Answer

Set the 'RedshiftTempDir' parameter to a larger S3 bucket.