MLS-C01 Data Engineering Practice Test 18 — 15 Questions

Question 1

Refer to the exhibit. A data engineer is troubleshooting an AWS Glue job that fails with an 'AccessDenied' error when trying to write to the S3 bucket 'my-data-lake'. The IAM policy attached to the Glue service role is shown. What is the missing permission?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

s3:ListBucket. The policy allows s3:GetObject and s3:PutObject on the bucket's objects, but it does not allow s3:ListBucket on the bucket itself. Many Glue operations require ListBucket to discover objects. Option A (s3:ListBucket) is the missing permission. Option B (s3:PutObjectAcl) is not needed. Option C (s3:GetBucketLocation) is not required. Option D (s3:DeleteObject) is not needed.

Answer

s3:PutObjectAcl

Answer

s3:GetBucketLocation

Answer

s3:DeleteObject

Question 2

A company is streaming real-time sensor data from IoT devices to Amazon Kinesis Data Streams. The data is then consumed by an AWS Lambda function that enriches the records with metadata from an Amazon DynamoDB table and writes the results to an Amazon S3 bucket. Recently, the Lambda function has been failing with 'ProvisionedThroughputExceededException' errors from DynamoDB. The data volume is variable, with occasional bursts. Which solution should a data engineer implement to resolve this issue without losing data?

Accepted Answer

Enable DynamoDB auto scaling for the table to automatically adjust read capacity based on demand.. DynamoDB auto scaling dynamically adjusts the table's provisioned read capacity based on actual traffic patterns, handling bursty sensor data without manual intervention. This prevents ProvisionedThroughputExceededExceptions while ensuring no data loss, as the Lambda function can retry failed operations. Auto scaling is the most cost-effective and operationally efficient solution for variable workloads.

Answer

Increase the DynamoDB table's provisioned read capacity units to a high static value.

Answer

Use an Amazon SQS queue to buffer the Lambda requests before querying DynamoDB.

Answer

Configure an Amazon SNS topic to throttle the data stream before it reaches Lambda.

Question 3

An e-commerce company uses Amazon Redshift for analytics. The data engineering team needs to load daily sales data from an S3 bucket that receives new files every hour. The data must be loaded into Redshift with minimal impact on query performance during the day, and they need to handle late-arriving data (files that appear after the daily load). Which approach should they use?

Accepted Answer

Use a staging table to load data incrementally with a MERGE operation, and schedule a late-arriving data job to merge files that arrive after the daily load.. It uses a staging table to incrementally load data with a MERGE operation, which minimizes impact on query performance by avoiding full table overwrites. The separate late-arriving data job handles files that appear after the daily load, ensuring completeness without blocking ongoing queries. This approach aligns with Redshift's best practices for incremental loads and late-arriving data handling.

Answer

Use AWS Glue ETL to copy the data from S3 to Redshift, overwriting the existing data each day.

Answer

Stream the data from S3 using Amazon Kinesis Firehose to load into Redshift continuously.

Answer

Use Amazon Redshift Spectrum to query data directly from S3 and create external tables.

Question 4

A data scientist needs to train a machine learning model using a large dataset (500 GB) stored in an S3 bucket. The training will be performed on a SageMaker notebook instance. The data scientist wants to minimize data transfer costs and reduce training time. Which data ingestion approach should the data engineer recommend?

Accepted Answer

Use the SageMaker SDK to directly read the data from S3 during training without copying it to the notebook.. The SageMaker SDK allows training jobs to read data directly from S3 using the Pipe or File mode, which avoids copying the 500 GB dataset to the notebook instance's EBS volume. This minimizes data transfer costs (no egress from S3 to the notebook) and reduces training time by streaming data directly to the training container without intermediate storage.

Answer

Copy the dataset to the notebook instance's attached EBS volume before training.

Answer

Load the dataset into an Amazon RDS database and query it from the notebook.

Answer

Mount the S3 bucket to the notebook instance using Amazon Elastic File System (EFS).

Question 5

A company is using AWS Glue to run ETL jobs that transform data from multiple sources into a data lake on S3. The jobs are scheduled to run hourly. Recently, the jobs have been failing intermittently with 'MemoryError' exceptions. The data volume has grown over time. The data engineer needs to resolve this issue cost-effectively. Which action should be taken?

Accepted Answer

Increase the number of DPUs allocated to the Glue job and use a larger worker type.. The 'MemoryError' exception indicates that the Glue job is running out of memory as data volume grows. Increasing the number of DPUs (Data Processing Units) and using a larger worker type (e.g., from Standard to G.1X or G.2X) provides more memory and compute capacity per worker, allowing the job to handle larger datasets without failing. This is the most cost-effective approach because it scales resources only as needed, avoiding over-provisioning.

Answer

Increase the S3 timeout settings in the Glue job configuration.

Answer

Switch the Glue job type from Spark to Python shell to reduce memory overhead.

Answer

Repartition the data using Spark's repartition method before processing.

Question 6

A data engineering team needs to set up a data pipeline that ingests streaming data from an Apache Kafka cluster running on Amazon EKS into an S3 data lake. The data must be stored in Parquet format, partitioned by date and event type. The team wants a fully managed solution with minimal operational overhead. Which solution should they choose?

Accepted Answer

Use Amazon MSK (Managed Streaming for Apache Kafka) and configure an MSK Connect S3 sink connector.. Amazon MSK is a fully managed Apache Kafka service that integrates with MSK Connect, which provides a pre-built S3 sink connector. This connector can directly stream data from Kafka topics to S3 in Parquet format with partitioning by date and event type, requiring no custom code or infrastructure management. This minimizes operational overhead while meeting all requirements for a fully managed solution.

Answer

Set up a Kinesis Data Firehose delivery stream that reads from Kafka and writes to S3.

Answer

Use AWS Glue ETL jobs to pull data from Kafka cluster periodically.

Answer

Create a Kinesis Data Analytics application to read from Kafka and write to S3.

Question 7

A data scientist is training a deep learning model using a large dataset stored in S3. The training job runs on a SageMaker training instance with a GPU. The data engineer notices that the GPU utilization is low, and the training is I/O bound. The data is read directly from S3 using the SageMaker SDK. Which change should the data engineer recommend to improve GPU utilization?

Accepted Answer

Use SageMaker Pipe mode to stream data directly from S3 to the training container.. SageMaker Pipe mode streams data directly from S3 to the training container, eliminating the need to download the entire dataset to disk. This reduces I/O latency and keeps the GPU fed with data, improving utilization. The current I/O bottleneck occurs because the SDK reads data from S3 as files, causing the GPU to wait for data.

Answer

Increase the batch size in the training script to process more data per step.

Answer

Mount the S3 bucket to the training instance using Amazon Elastic File System (EFS).

Answer

Copy the entire dataset to an Amazon EBS volume attached to the training instance.

Question 8

A company uses Amazon DynamoDB as the primary data store for a real-time recommendation engine. The data engineering team needs to export a daily snapshot of the DynamoDB table to S3 for offline analytics. The table is large (10 TB) and has a high read/write throughput. Which method will export the data with the least impact on the production workload?

Accepted Answer

Use the DynamoDB export to S3 feature available in the AWS Console or CLI.. The native DynamoDB export to S3 feature uses the table's internal backup mechanism (point-in-time recovery) to export data without consuming any read capacity units (RCUs) from the production table. This ensures zero impact on the live workload, even for a 10 TB table with high throughput.

Answer

Use AWS Data Pipeline to export the DynamoDB table to S3.

Answer

Use DynamoDB Scan API with parallel scans to export data to S3.

Answer

Use AWS Glue ETL job with a DynamoDB connection to export data.

Question 9

A data engineer is building a data pipeline that uses AWS Lambda to process records from an SQS queue and write results to an S3 bucket. The Lambda function processes each record individually and writes a separate file to S3. The team notices high latency and wants to reduce the number of S3 PUT requests to improve performance and reduce cost. Which approach should the data engineer take?

Accepted Answer

Aggregate multiple records into a single file in a DynamoDB table, then periodically write the aggregated data to S3.. It reduces the number of S3 PUT requests by aggregating multiple records into a single file in DynamoDB and then periodically writing the aggregated data to S3. This approach directly addresses the high latency and cost issue caused by writing a separate S3 object per record, as S3 PUT requests are billed per operation and have overhead. By batching records before writing, the pipeline reduces the total number of PUT requests, improving throughput and lowering costs.

Answer

Use S3 multipart upload for each record to improve throughput.

Answer

Increase the Lambda function's memory allocation to improve processing speed.

Answer

Use S3 Batch Operations to process the records in batches.

Question 10

A company has a large number of small CSV files (hundreds of thousands) in an S3 bucket. A data engineer needs to run a SQL query on this data using Amazon Athena. The queries are currently slow and expensive. Which two actions will improve query performance and reduce cost?

Accepted Answer

Partition the data by a commonly filtered column (e.g., date).. The correct answers are C and E. Partitioning the data by a commonly filtered column (e.g., date) reduces the amount of data scanned by Athena, improving performance and cost. Converting the data to Parquet or ORC columnar format further reduces the data scanned and improves compression and query speed. Option B (gzip compression) helps reduce storage and scan volume but is less impactful than partitioning and columnar format. Option A (increasing S3 request rate) does not directly improve Athena query performance. Option D (splitting into smaller files) can increase the overhead of reading many small files, potentially hurting performance.

Answer

Increase the S3 request rate per prefix to improve read throughput.

Answer

Compress the CSV files using gzip.

Answer

Increase the number of partitions by splitting files into smaller ones.

Question 11

A data engineer needs to design a data ingestion pipeline that ingests data from a MySQL database hosted on-premises into Amazon S3 for analytics. The pipeline must capture change data (CDC) and run continuously with low latency. Which two services should the data engineer use?

Accepted Answer

AWS Database Migration Service (DMS) with ongoing replication.. AWS Database Migration Service (DMS) with ongoing replication can continuously capture changes from on-premises MySQL using Change Data Capture (CDC). Amazon S3 can be configured as the target endpoint for DMS, allowing the CDC data to be written directly to S3 with low latency. Option C (Amazon AppFlow) is designed for SaaS applications, not on-premises databases. Option D (AWS Glue ETL) is batch-oriented and not suitable for low-latency continuous ingestion. Option E (Amazon Kinesis Data Streams) is not required because DMS can directly write to S3.

Answer

Amazon AppFlow.

Answer

AWS Glue ETL jobs scheduled at regular intervals.

Answer

Amazon Kinesis Data Streams.

Question 12

A company is using Amazon Redshift for data warehousing. The data engineering team observes that query performance degrades over time due to data skew. Which three strategies should the team implement to improve performance?

Accepted Answer

Choose appropriate distribution keys based on join and group-by columns.. Choosing appropriate distribution keys based on join and group-by columns minimizes data movement across nodes during query execution. In Amazon Redshift, data is distributed across compute nodes according to the distribution key; aligning it with frequently joined or aggregated columns ensures that related rows are co-located on the same slice, reducing network shuffling and improving query performance.

Answer

Increase the number of nodes in the Redshift cluster.

Answer

Drop unused indexes on large tables.

Question 13

Refer to the exhibit. An IAM policy is attached to a data engineering team's role. The team needs to upload data to the 'confidential' prefix in the 'my-data-lake' bucket. However, they are receiving 'AccessDenied' errors. What is the likely cause?

Accepted Answer

The Deny statement with the condition explicitly denies access to the 'confidential' prefix for accounts other than 123456789012.. The Deny statement explicitly denies access to the 'confidential' prefix when the request comes from an AWS account other than 123456789012. Since the data engineering team's role likely belongs to a different account, the Deny condition matches and overrides any Allow statements, resulting in an 'AccessDenied' error. In IAM, an explicit Deny always takes precedence over an Allow, so even if the Allow statement grants PutObject, the Deny blocks the upload.

Answer

The condition in the Deny statement requires the team to use a specific source IP address.

Answer

The Allow statement only grants GetObject and PutObject, but the team needs ListBucket.

Answer

The Allow statement's resource does not include the 'confidential' prefix.

Question 14

Refer to the exhibit. A data engineer runs an Athena query and gets a failure. What is the most likely cause?

Accepted Answer

The IAM role used does not have permissions to write to S3.. The most likely cause for an Athena query failure is that the IAM role used does not have the necessary permissions to write query results to the specified S3 bucket. While syntax errors or encryption settings can cause issues, permissions are a frequent and common cause of failure in practice.

Answer

The query result location uses a bucket with default encryption enabled.

Answer

The SQL query syntax is incorrect.

Answer

The output S3 bucket specified in the query result configuration already exists.

Question 15

Refer to the exhibit. A data engineer has deployed this CloudFormation template. The Glue job 'my-etl-job' reads from the S3 bucket 'my-data-lake-bucket' and writes transformed data to another bucket. After 30 days, the data engineer notices that the Glue job fails with 'Input data not found' errors. What is the most likely cause?

Accepted Answer

The lifecycle configuration deletes objects from the bucket after 30 days, removing the input data.. The lifecycle configuration on the S3 bucket 'my-data-lake-bucket' is set to delete objects after 30 days. Since the Glue job 'my-etl-job' reads input data from this bucket, once the 30-day period elapses, the input data is removed, causing the 'Input data not found' error. This matches the symptom of the job failing after exactly 30 days.

Answer

The temporary directory 'my-temp-dir' is being cleaned up by the lifecycle configuration.

Answer

The script location 's3://my-scripts/etl.py' is being deleted by the lifecycle rule.

Answer

The job bookmark option 'job-bookmark-enable' is causing the job to skip newly arriving data.