MLS-C01 Data Engineering Practice Test 4 — 15 Questions

Question 1

A company uses AWS Glue to catalog data in S3. Data is partitioned by year, month, day. The Glue crawler runs daily but sometimes misses new partitions. What should be done to ensure all partitions are cataloged?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Configure the crawler to update all partitions on each run.. Configuring the Glue crawler to 'update all partitions on each run' forces the crawler to scan the entire S3 path and register any new partitions it finds, even if the partition structure hasn't changed. This ensures all missed partitions are cataloged. Partition indexing (D) improves query performance by creating an index over existing partitions, but does not automatically discover new partitions.

Answer

Use a custom classifier to detect partition patterns.

Answer

Increase the crawler schedule to run every hour.

Answer

Enable partition indexing in the Glue table properties.

Question 2

A data engineer is designing a streaming pipeline using Amazon Kinesis Data Analytics for Apache Flink. The pipeline reads from a Kinesis data stream and writes to a S3 bucket. The job must recover quickly from failures without reprocessing large amounts of data. Which TWO configurations should be used? (Choose TWO)

Accepted Answer

Enable checkpointing with a state backend like RocksDB.. Enabling checkpointing with a state backend like RocksDB allows Apache Flink to periodically save the state of the streaming application to durable storage. In the event of a failure, Flink can restart from the last completed checkpoint, avoiding the need to reprocess large amounts of data from the beginning of the stream. RocksDB is specifically designed for large state and provides fast recovery by storing state on disk with memory caching, making it ideal for production streaming pipelines.

Answer

Use in-memory state backend for low latency.

Answer

Set the parallelism to the maximum number of shards.

Answer

Increase the retention period of the Kinesis stream to 365 days.

Question 3

A company needs to build a data lake on AWS for analytics. The data includes structured, semi-structured, and unstructured data. The solution must support schema-on-read, provide fine-grained access control, and be cost-effective for storing rarely accessed data. Which THREE services should be used? (Choose THREE)

Accepted Answer

AWS Glue Data Catalog for schema-on-read.. AWS Glue Data Catalog is correct because it provides a centralized metadata repository that enables schema-on-read for data stored in Amazon S3. It allows you to define table schemas and partitions without transforming the underlying data, so analytics tools like Amazon Athena and Amazon EMR can query the data with the schema applied at read time.

Answer

Amazon Redshift for data warehousing.

Answer

Amazon EMR for data processing.

Question 4

A data engineer created an IAM policy to allow a Glue ETL job to read and write objects to an S3 bucket. The ETL job fails when writing data with the error 'Access Denied'. The job is configured to use SSE-S3 (AES256) encryption. What is the likely issue?

Accepted Answer

The policy does not grant s3:PutObject on the bucket itself, which is needed for some write operations.. The error 'Access Denied' when writing to S3 with SSE-S3 encryption typically occurs because the IAM policy lacks the `s3:PutObject` permission on the bucket resource itself. While the policy may grant `s3:PutObject` on the object ARN (`arn:aws:s3:::bucket/*`), some S3 write operations—especially those involving encryption headers or bucket-level checks—also require the permission on the bucket ARN (`arn:aws:s3:::bucket`). Without this, the request is denied even if the object-level permission exists.

Answer

The policy grants s3:PutObject on all buckets, not just the specific one.

Answer

The condition requires objects to be encrypted with SSE-KMS, but the job uses SSE-S3.

Answer

The condition requires objects to use SSE-S3, but the job uses SSE-KMS.

Question 5

A company runs a real-time fraud detection system using Amazon Kinesis Data Streams with 100 shards. Data is consumed by a custom Java application running on Amazon EC2 instances in an Auto Scaling group. The application processes records and writes results to a DynamoDB table. Over the past month, the application has experienced intermittent slowdowns and the DynamoDB write capacity has been fully utilized during peak hours. The team wants to improve throughput without losing the ability to reprocess failed records. The application currently uses the Kinesis Client Library (KCL) with DynamoDB as the lease table. The team is considering the following changes: A. Increase the number of EC2 instances to match the number of shards. B. Switch to using AWS Lambda as the consumer to handle scaling automatically. C. Increase the write capacity of the DynamoDB lease table to handle more workers. D. Use enhanced fan-out to have each consumer receive its own 2 MB/second shard throughput. Which change should the team implement first to address the issue?

Accepted Answer

Use enhanced fan-out to have each consumer receive its own 2 MB/second shard throughput.. The primary bottleneck is DynamoDB write capacity being fully utilized during peak hours. Enhanced fan-out (option B) provides each consumer with a dedicated 2 MB/second read throughput per shard, eliminating the need for consumers to contend for the shared 2 MB/second per shard. This reduces the load on the DynamoDB lease table because workers no longer need to poll for records, which in turn lowers the write operations to the lease table and alleviates the DynamoDB write capacity issue.

Answer

Increase the write capacity of the DynamoDB lease table to handle more workers.

Answer

Switch to using AWS Lambda as the consumer to handle scaling automatically.

Answer

Increase the number of EC2 instances to match the number of shards.

Question 6

A retail company runs an e-commerce platform on AWS. They have a Data Engineering team that processes clickstream data using Amazon Kinesis Data Streams (KDS) with a shard count of 5. The data is consumed by an AWS Lambda function that transforms and loads the data into an Amazon S3 bucket partitioned by year/month/day/hour. Recently, the team has noticed that the Lambda function is experiencing throttling errors, and the KDS shard iterator age is increasing, indicating that the consumer cannot keep up with the incoming data rate. The team has already increased the Lambda reserved concurrency to 1000 and enabled batch window of 60 seconds. The metrics show that the Lambda function duration is well under the 5-minute timeout, and there are no errors in the transformation logic. The S3 write operations are not failing. Which course of action would MOST effectively resolve the issue without unnecessary cost or complexity?

Accepted Answer

Increase the number of shards in the Kinesis Data Stream to 20 to increase the parallelism of Lambda consumers.. The core issue is that the Lambda consumer cannot keep up with the incoming data rate, as evidenced by the increasing shard iterator age. Increasing the shard count from 5 to 20 directly increases the number of Kinesis Data Streams shards, which in turn increases the number of concurrent Lambda invocations (one per shard) and the overall throughput of the stream. This addresses the bottleneck at the source without adding unnecessary complexity or cost, as KDS pricing is based on shard hours and Lambda concurrency is already set to 1000.

Answer

Increase the Lambda reserved concurrency to 5000 to allow more parallel executions.

Answer

Increase the batch window to 300 seconds to accumulate more records per invocation and reduce the number of calls.

Answer

Switch to using Amazon Kinesis Data Analytics with a larger instance type to process the stream.

Question 7

Drag and drop the steps to create an Amazon SageMaker notebook instance in the correct order.

Accepted Answer

First, open Amazon SageMaker console. Next, select 'Notebook instances' from the left navigation. Then, click 'Create notebook instance'. After that, configure the notebook instance settings (name, instance type, IAM role, VPC). Finally, click 'Create notebook instance'.. Creating a notebook instance requires navigating the SageMaker console, configuring instance settings, IAM role, and VPC, then launching.

Answer

First, open Amazon SageMaker console. Then, click 'Create notebook instance' immediately. Next, configure the notebook instance settings. Finally, select 'Notebook instances' from the navigation.

Answer

First, configure the notebook instance settings (name, instance type, IAM role, VPC). Then, open Amazon SageMaker console. Next, click 'Create notebook instance'. Finally, create the notebook instance.

Answer

First, click 'Create notebook instance'. Then, open Amazon SageMaker console. Next, select 'Notebook instances'. Finally, configure settings and create.

Question 8

Drag and drop the steps to perform hyperparameter tuning using SageMaker Automatic Model Tuning in the correct order.

Accepted Answer

Define the objective metric, then define the hyperparameter search space, then configure resource limits, then create the tuning job, then monitor and select the best model.. Tuning involves defining search space, creating a tuning job, setting limits, executing, and selecting best model.

Answer

Define the hyperparameter search space, then configure resource limits, then define the objective metric, then create the tuning job, then monitor and select the best model.

Answer

Select the best model, then define the objective metric, then define the hyperparameter search space, then configure resource limits, then create the tuning job.

Answer

Create the tuning job, then define the objective metric, then define the hyperparameter search space, then configure resource limits, then monitor and select the best model.

Question 9

Drag and drop the steps to set up Amazon SageMaker Ground Truth for a labeling job in the correct order.

Accepted Answer

Prepare dataset, Create labeling job, Configure task settings, Provide labeling instructions, Execute labeling job. Ground Truth setup involves dataset preparation, job creation, task configuration, instructions, and execution.

Answer

Prepare dataset, Create labeling job, Provide labeling instructions, Configure task settings, Execute labeling job

Answer

Create labeling job, Prepare dataset, Configure task settings, Provide labeling instructions, Execute labeling job

Answer

Prepare dataset, Create labeling job, Configure task settings, Execute labeling job, Provide labeling instructions

Question 10

Drag and drop the steps to set up cross-validation in a SageMaker training job using the built-in XGBoost algorithm in the correct order.

Accepted Answer

Split the data into training and validation folds, then configure the SageMaker estimator with cross-validation parameters (e.g., num_round, eval_metric, num_cv_folds), then execute the training job (fit), then select the best model based on average CV metrics.. Cross-validation requires data splitting, job configuration with CV parameters, execution, and model selection.

Answer

Configure the SageMaker estimator with cross-validation parameters, then split the data into folds, then execute the training job, then select the best model.

Answer

Split the data into folds, then execute the training job, then configure the estimator, then select the best model.

Answer

Execute the training job, then split the data into folds, then configure the estimator, then select the best model.

Question 11

Match each AWS service to its primary purpose in a machine learning pipeline.

Accepted Answer

Amazon SageMaker: Fully managed service for building, training, and deploying machine learning models.. The correct matches associate each service with its primary ML pipeline role. SageMaker is for end-to-end ML, Lambda for serverless inference, S3 for data/model storage, Glue for data preparation. Distractors confuse these roles.

Answer

Amazon SageMaker: Serverless compute service for inference.

Answer

AWS Lambda: Scalable object storage for models.

Question 12

Match each AWS security service to its function in ML.

Accepted Answer

AWS KMS: Manages encryption keys for securing ML model data at rest and in transit. In ML, AWS KMS handles encryption keys, IAM manages access, CloudTrail provides audit logs. Common confusions arise because these services often work together, but each has a distinct primary function.

Answer

AWS KMS: Provides identity-based access control for ML resources

Answer

AWS IAM: Encrypts data during ML processing

Answer

AWS CloudTrail: Monitors and enforces compliance rules for ML

Question 13

Match each data format to its typical use in AWS ML.

Accepted Answer

CSV: Tabular data for built-in algorithms. Common data formats in AWS ML: CSV for tabular data, JSON for semi-structured inference I/O, Parquet for columnar analytics, and TFRecord for TensorFlow training. Distractors confuse CSV with Parquet and JSON with binary formats.

Answer

CSV: Columnar storage for analytics

Answer

JSON: Binary format for efficient I/O

Question 14

Match each SageMaker optimization technique to its description.

Accepted Answer

Managed Spot Training: Uses EC2 Spot instances to reduce training cost. Managed Spot Training reduces cost via spot instances; Data Parallelism speeds training by distributing data; Automatic Model Tuning finds optimal hyperparameters; Compilation (Neo) optimizes for inference hardware. Common confusions include swapping definitions between these techniques.

Answer

Managed Spot Training: Optimizes model for inference on edge devices

Answer

SageMaker Data Parallelism: Automatically searches hyperparameter space to optimize model

Question 15

A data engineering team needs to process streaming data from thousands of IoT devices. They want to aggregate data in 1-minute windows and store results in an S3 data lake for downstream analytics. Which architecture should they use?

Accepted Answer

Use Kinesis Data Analytics for SQL-based windowed aggregations and send results to Kinesis Data Firehose for delivery to S3.. Kinesis Data Analytics for SQL Applications is purpose-built for real-time windowed aggregations on streaming data, such as 1-minute tumbling windows. It can directly consume from Kinesis Data Streams, perform the aggregation using standard SQL, and output the results to Kinesis Data Firehose, which reliably delivers the aggregated data to an S3 data lake with built-in buffering and compression.

Answer

Use AWS Glue ETL jobs running in streaming mode to read from Kinesis Data Streams, apply window aggregations, and write to S3.

Answer

Use Kinesis Data Streams with enhanced fan-out and multiple consumers to aggregate windows, then write to S3 via Firehose.

Answer

Use Kinesis Data Streams, trigger a Lambda function for 1-minute window aggregation using Python, and write results to S3.