DEA-C01 Data Operations and Support Practice Test 2 — 15 Questions

Question 1

A company runs a data pipeline that ingests clickstream data from a web application into Amazon Kinesis Data Streams. A Lambda function processes records from the stream and writes them to an Amazon S3 bucket in JSON format. The pipeline has been running smoothly, but for the past hour, the Lambda function has been failing with 'Rate exceeded' errors, and the Kinesis stream shows elevated 'IteratorAgeMilliseconds' metrics. The Lambda function has a reserved concurrency of 100, and the Kinesis stream has 10 shards. The average record size is 5 KB, and the data rate is approximately 15 MB per second. Which combination of actions should a data engineer take to resolve the issue and prevent recurrence? (Choose TWO.)

Accepted Answer

Increase the Lambda function's reserved concurrency to 200.. The 'Rate exceeded' errors indicate that the Lambda function's concurrency is insufficient to keep up with the incoming data rate from Kinesis. With 10 shards and a 15 MB/s data rate, each shard processes ~1.5 MB/s, and with 5 KB records, that's ~300 records per second per shard. Increasing reserved concurrency to 200 allows more parallel invocations to handle the load, reducing the iterator age.

Answer

Decrease the Lambda function's batch size from 100 to 50.

Answer

Enable S3 multipart upload for the Lambda function.

Answer

Replace the Lambda function with Amazon Kinesis Data Firehose to write directly to S3.

Question 2

A company uses AWS Glue to run ETL jobs that process data from Amazon S3 and write results to Amazon Redshift. The Glue job uses the JDBC connection to Redshift. Recently, the job has been failing intermittently with the error: 'java.sql.SQLException: [Amazon](500310) Invalid operation: INSERT has more expressions than target columns;' The Glue job writes to a staging table in Redshift before performing a merge into the final table. The staging table schema matches the source data. The error occurs only on some days and affects different columns each time. The data engineer suspects that the source data occasionally contains extra columns due to a schema drift in the upstream data producer. Which approach should the data engineer take to handle this issue robustly?

Accepted Answer

Use a Glue DynamicFrame and apply the resolveChoice method to make the schema consistent.. Option B is correct because Glue DynamicFrames can automatically handle schema drift using the `resolveChoice` method, which allows you to specify how to handle columns that appear inconsistently across records (e.g., making them null, casting to a common type, or dropping them). This directly addresses the intermittent error caused by extra columns in the source data without requiring manual schema updates or fragile conditional logic.

Answer

Skip any records that have extra columns by adding a conditional check in the Glue script.

Answer

Manually update the Redshift staging table schema whenever the source data changes.

Answer

Use a Glue DynamicFrame and apply the dropFields method to remove extra columns before writing.

Question 3

A data engineer is troubleshooting a Glue ETL job that reads from an S3 bucket and writes to a Redshift table. The job fails with a 'MemoryError' when processing a large dataset. Which TWO actions should the engineer take to resolve this issue? (Choose TWO.)

Accepted Answer

Increase the number of DPUs and set 'spark.sql.shuffle.partitions' to a higher value.. Option A is correct because increasing the number of DPUs (Data Processing Units) provides more memory and compute resources to the Glue job, directly addressing the MemoryError. Setting 'spark.sql.shuffle.partitions' to a higher value reduces the amount of data shuffled per partition, preventing out-of-memory errors during wide transformations like joins or aggregations.

Answer

Increase the number of DPUs and set 'coalesce(1)' in the script.

Answer

Decrease the number of DPUs and increase 'spark.shuffle.partitions'.

Answer

Set the 'RedshiftTempDir' parameter to a larger S3 bucket.

Question 4

A data engineer applies the above S3 bucket policy to an S3 bucket used by a Glue ETL job. The Glue job writes objects to the bucket. Which of the following is true about the behavior of the policy?

Accepted Answer

The policy denies all PutObject requests because the Allow and Deny statements are contradictory.. The correct answer is C because in AWS IAM policy evaluation, an explicit Deny always overrides any Allow. The policy has an Allow statement granting s3:PutObject for all principals, but a separate Deny statement explicitly denies s3:PutObject when the encryption condition is not aws:kms. Since the Deny applies to all PutObject requests (including those with no encryption or AES256), and the Allow does not include a condition to match only aws:kms, the Deny takes precedence and blocks all PutObject requests, making the policy effectively deny all PutObject operations.

Answer

The policy allows PutObject with aws:kms encryption because the Allow statement is broader.

Answer

The policy allows PutObject with no encryption because the Deny only applies to PutObject.

Answer

The policy allows PutObject with AES256 encryption and denies PutObject with aws:kms encryption.

Question 5

A company runs a nightly batch processing pipeline using AWS Glue ETL jobs. The pipeline reads data from an Amazon S3 bucket, transforms it, and writes results to an Amazon Redshift cluster. Recently, the data volume has increased significantly, and some Glue jobs are failing with the error 'java.lang.OutOfMemoryError: Java heap space'. The data engineer needs to modify the job configuration to prevent these failures without changing the code. The job currently uses 10 DPUs and processes data in a single Spark DataFrame. Which of the following is the MOST effective solution?

Accepted Answer

Increase the number of DPUs to 20 and enable job bookmarking for incremental processing.. Increasing DPUs from 10 to 20 provides more memory and compute resources, directly addressing the 'java.lang.OutOfMemoryError: Java heap space' caused by insufficient memory for the single DataFrame. Enabling job bookmarking allows incremental processing, which reduces the volume of data processed per run, further mitigating memory pressure without code changes.

Answer

Reduce the number of DPUs to 5 and increase the Spark executor memory by setting 'spark.executor.memory' in job parameters.

Answer

Change the script to use DynamicFrame instead of DataFrame and disable the 'spark.sql.shuffle.partitions' configuration.

Answer

Add a 'coalesce(1)' operation before writing to Redshift to reduce the number of output files.

Question 6

Order the steps to set up a Kinesis Data Analytics application for real-time stream processing.

Question 7

Arrange the steps to set up a streaming ETL pipeline using Amazon Kinesis Data Firehose to Amazon S3.

Question 8

Match each AWS data compression format to its typical use case.

Question 9

Match each AWS data analytics service to its primary function.

Question 10

A data engineer notices that a nightly AWS Glue ETL job has been failing for the past three days with the error 'Unable to locate credentials'. The job uses an IAM role for execution. What is the most likely cause of this error?

Accepted Answer

The IAM role's trust policy does not include glue.amazonaws.com as a trusted entity.. The error 'Unable to locate credentials' indicates that the AWS Glue job cannot obtain AWS credentials to authenticate API calls. Since the job uses an IAM role for execution, the most likely cause is that the trust policy of that IAM role does not include 'glue.amazonaws.com' as a trusted entity. Without this trust relationship, AWS Glue cannot assume the role and thus has no credentials to sign requests.

Answer

The IAM role does not have an access key attached.

Answer

The S3 bucket name in the job parameters is misspelled.

Answer

The JDBC connection string contains an incorrect password.

Question 11

A company runs a data pipeline using AWS Lambda to process records from an Amazon Kinesis Data Stream. Recently, the Lambda function has been experiencing high invocation errors and the stream is throttling. The function performs simple transformations and writes to Amazon S3. What is the most effective way to reduce throttling and errors?

Accepted Answer

Increase the batch size in the Lambda event source mapping.. Increasing the batch size in the Lambda event source mapping allows each invocation to process more records from the Kinesis stream, reducing the number of total invocations. This lowers the rate at which Lambda polls the stream, which decreases the likelihood of hitting the Kinesis read throughput limits (5 transactions per second per shard) and reduces throttling errors. The simple transformations and S3 writes are likely I/O-bound, so larger batches improve throughput without increasing invocation concurrency.

Answer

Increase the Lambda function timeout.

Answer

Enable provisioned concurrency on the Lambda function.

Answer

Increase the number of shards in the Kinesis stream.

Question 12

A data engineer is troubleshooting a slow Amazon Redshift query that joins a large fact table with several dimension tables. The EXPLAIN plan shows a hash join on the distribution key, but the query still runs slowly. The fact table is distributed by KEY(column_x) and the dimension tables are distributed ALL. The engineer notices that the fact table has a high number of rows with the same value in column_x. What is the most likely cause of the slow performance?

Accepted Answer

The fact table's distribution key column has data skew, causing uneven data distribution across nodes.. Option A is correct because data skew in the distribution key column_x causes some slices to hold a disproportionate number of rows, leading to uneven workload distribution during the hash join. The EXPLAIN plan shows a hash join on the distribution key, which should be efficient if data is evenly distributed, but skew forces the node with the most rows to become a bottleneck, slowing the entire query.

Answer

The dimension tables should be distributed by KEY instead of ALL.

Answer

The Redshift cluster does not have enough disk space.

Answer

The fact table does not have a sort key.

Question 13

A company uses Amazon S3 to store large CSV files and runs Amazon Athena queries on them. The queries are becoming slower as data grows. A data engineer suggests converting the files to Apache Parquet format and partitioning the data. What is the primary benefit of converting to Parquet?

Accepted Answer

Parquet stores data in a columnar format, reducing the amount of data scanned per query.. Parquet is a columnar storage format that stores data by columns rather than rows. When Athena queries only a subset of columns, it can read just those columns from disk, drastically reducing the amount of data scanned per query. This directly addresses the performance slowdown because Athena charges by data scanned, and less scanning means faster queries and lower costs.

Answer

Parquet allows schema evolution without rewriting files.

Answer

Parquet supports nested data structures that CSV cannot.

Answer

Parquet is compressed by default, reducing storage costs.

Question 14

A data engineer is running an AWS Glue ETL job that reads from an Amazon RDS MySQL database and writes to Amazon S3. The job fails with a 'Communications link failure' error. The security group for the RDS instance allows inbound traffic from the Glue job's security group. What is the most likely cause of the failure?

Accepted Answer

The Glue job's security group does not allow outbound traffic to the RDS security group on port 3306.. Option C is correct because AWS Glue ETL jobs run in a VPC that requires outbound security group rules to initiate connections to RDS. Even if the RDS security group allows inbound traffic from the Glue security group, the Glue security group must also have an outbound rule allowing traffic to the RDS security group on port 3306 (MySQL default port). Without this outbound rule, the TCP handshake from Glue to RDS fails, causing a 'Communications link failure'.

Answer

The JDBC connection string in the Glue job does not include the database name.

Answer

The Glue job is using the wrong JDBC driver.

Answer

The IAM role used by the Glue job does not have rds:Connect permission.

Question 15

A company runs a critical data pipeline using Amazon Kinesis Data Firehose to deliver data to Amazon S3. The data is compressed with GZIP and partitioned by year/month/day/hour. Recently, the delivery to S3 has been failing with 'Rate exceeded' errors. The Firehose delivery stream has a buffer size of 128 MB and buffer interval of 60 seconds. What is the most effective way to resolve this issue?

Accepted Answer

Increase the buffer size to 256 MB and buffer interval to 120 seconds.. The 'Rate exceeded' error indicates that Kinesis Data Firehose is sending requests to S3 at a rate that exceeds the S3 bucket's request rate limits for PUT operations. Increasing the buffer size to 256 MB and the buffer interval to 120 seconds allows Firehose to accumulate more data before each S3 PUT request, reducing the number of requests per second and staying within S3's 3,500 PUT requests per second limit per prefix. This directly addresses the throttling issue without changing the data volume.

Answer

Transition objects to S3 Glacier after 30 days.

Answer

Decrease the buffer size to 64 MB and buffer interval to 30 seconds.

Answer

Enable server-side encryption on the S3 bucket.