MLS-C01 Machine Learning Implementation and Operations Practice Test 13 — 15 Questions

Question 1

A data scientist is training a model using Amazon SageMaker and wants to monitor the training job's resource utilization and metrics in real time. Which AWS service should the data scientist use to view metrics such as CPU utilization and memory usage for the training job?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Amazon CloudWatch. Amazon CloudWatch is the primary service for monitoring AWS resources, including SageMaker training jobs. It automatically collects metrics such as CPU utilization, memory usage, and disk I/O from the training instances. These metrics can be viewed in the CloudWatch console or used to create alarms. Other services like CloudTrail, Trusted Advisor, and SageMaker Debugger serve different purposes and do not provide the required real-time resource monitoring.

Answer

AWS Trusted Advisor

Answer

AWS CloudTrail

Answer

Amazon SageMaker Debugger

Question 2

A machine learning engineer trained a model with Amazon SageMaker and needs to automatically stop training if the validation loss stops improving for 5 consecutive epochs to avoid overfitting and save cost. Which SageMaker training configuration should be used?

Accepted Answer

Enable early stopping in the SageMaker training job by setting the stopping condition in the algorithm's hyperparameters or using the SageMaker Debugger built-in rule.. Early stopping in SageMaker is implemented via algorithm hyperparameters (e.g., early_stopping for built-in algorithms) or SageMaker Debugger rules that track metrics and terminate training when a condition like no improvement for N epochs is met. These mechanisms directly address the need to stop training to prevent overfitting and reduce cost.

Answer

Set the training job's volume size to 5 GB and enable network isolation.

Answer

Use SageMaker Automatic Model Tuning with a maximum number of training jobs set to 5.

Answer

Configure the training job's resource configuration to use spot instances with a maximum wait time of 5 epochs.

Question 3

A company is using Amazon SageMaker to train a model on a dataset that contains sensitive information. The company must ensure that the data is encrypted at rest and in transit during training. The training job uses a built-in algorithm and stores model artifacts in Amazon S3. Which combination of configurations should the company use to meet these requirements?

Accepted Answer

Enable inter-container traffic encryption and specify a KMS key for volume encryption in the training job configuration.. To encrypt data at rest and in transit during SageMaker training, enable inter-container traffic encryption for in-transit protection and specify a KMS key for volume encryption for at-rest protection. These settings are available in the training job configuration and ensure comprehensive encryption.

Answer

Enable SageMaker Debugger and use AWS Certificate Manager (ACM) to encrypt training data.

Answer

Enable network isolation and use S3 server-side encryption with SSE-S3 for the training data.

Answer

Use a VPC configuration with a NAT gateway and enable S3 default encryption.

Question 4

A company is using Amazon SageMaker to train a model on a large dataset stored in Amazon S3. The training job is taking too long to download the data. The company wants to reduce the training time by improving data throughput from S3. Which SageMaker training feature should they enable?

Accepted Answer

Pipe input mode. Pipe input mode streams data from S3 on the fly, so the training job doesn't wait for the full dataset to download. This reduces the time to start training and can improve overall throughput, especially for large datasets. Other input modes either download all data first or don't directly address throughput.

Answer

Augmented manifest files

Answer

File input mode

Answer

SageMaker FastFile mode

Question 5

A machine learning engineer is using Amazon SageMaker to train a model on a large dataset. The training job is configured to use multiple instances for distributed training. The engineer notices that the training is not scaling efficiently and suspects communication overhead between instances. Which technique should the engineer use to reduce communication overhead and improve scaling?

Accepted Answer

Use SageMaker distributed model parallel library. Model parallelism partitions the model across instances, reducing the need for full gradient synchronization and lowering communication overhead. This is particularly effective for large models that cannot fit on a single instance. Data parallelism still requires synchronization, and increasing batch size or instance size does not directly address communication costs.

Answer

Use a larger instance type with more GPUs

Answer

Use SageMaker distributed data parallel library

Answer

Increase the batch size to reduce the number of gradient updates

Question 6

A machine learning engineer is deploying a real-time inference endpoint using Amazon SageMaker. The model is a TensorFlow model that requires 4 GB of memory and performs best on GPU. The endpoint must handle varying traffic patterns, with occasional spikes. The engineer wants to minimize cost while ensuring that the endpoint can scale automatically based on traffic. The engineer also wants to avoid cold starts that could add latency during spikes. Which solution meets these requirements?

Accepted Answer

Deploy the model to a SageMaker endpoint with an automatic scaling policy based on SageMakerVariantInvocationsPerInstance, with a minimum instance count of 2 and a maximum of 10, and enable scale-in cooldown to avoid rapid scale-in.. SageMaker automatic scaling uses metrics like SageMakerVariantInvocationsPerInstance to scale endpoints. Setting a minimum instance count of 2 provides baseline capacity to handle sudden spikes without cold starts, while a maximum of 10 controls cost. Scale-in cooldown prevents flapping. Using CPUUtilization is less effective for GPU workloads, and provisioned concurrency is not a SageMaker feature.

Answer

Deploy the model to a SageMaker endpoint with an automatic scaling policy based on CPUUtilization, with a minimum instance count of 1 and a maximum of 10.

Answer

Deploy the model to a SageMaker endpoint with a provisioned concurrency of 2 and automatic scaling based on Invocations.

Answer

Deploy the model to a SageMaker endpoint with an automatic scaling policy based on InvocationsPerInstance, with a minimum instance count of 1 and a maximum of 10.

Question 7

A company is using Amazon SageMaker to train a deep learning model for image classification. The training job is using a single p3.2xlarge instance and is taking too long. The data is stored in Amazon S3 as large TFRecord files. The company wants to reduce training time by using distributed training. Which approach should be used to minimize training time while maintaining model accuracy?

Accepted Answer

Use SageMaker distributed model parallel training to split the model across multiple instances.. SageMaker distributed model parallel training is specifically designed for large models that exceed the memory of a single GPU. By splitting the model across multiple instances, it enables training of larger models and reduces training time by leveraging additional compute resources. This approach is more suitable than data parallel training when the model itself is the bottleneck.

Answer

Use SageMaker Pipe mode to stream data from Amazon S3 to reduce I/O overhead.

Answer

Use SageMaker Automatic Model Tuning to find the best hyperparameters for faster convergence.

Answer

Use SageMaker distributed data parallel training with Horovod and increase the number of instances.

Question 8

A machine learning engineer has deployed a model to a SageMaker endpoint. The endpoint is configured with an auto scaling policy based on the SageMakerVariantInvocationsPerInstance metric. During a sudden traffic spike, the endpoint's latency increased significantly, and some requests timed out. The engineer noticed that the number of instances did not increase quickly enough to handle the spike. The engineer wants to reduce the time it takes for the endpoint to scale out during future spikes. Which action should the engineer take?

Accepted Answer

Decrease the scale-out cooldown period.. The scale-out cooldown period determines the delay before another scale-out activity can occur. Decreasing it allows the endpoint to add instances more quickly during a spike. Increasing scale-in cooldown affects scale-in, not scale-out. Increasing the target value makes scaling less sensitive, and switching to CPUUtilization may not reflect true load. Thus, decreasing scale-out cooldown is the correct action.

Answer

Increase the target value for SageMakerVariantInvocationsPerInstance.

Answer

Switch the scaling metric to CPUUtilization.

Answer

Increase the scale-in cooldown period.

Question 9

A company is using Amazon SageMaker to train a model with a custom algorithm. The training script reads data from an S3 bucket using boto3. The training job fails with an 'AccessDenied' error when trying to access the S3 bucket. The IAM role attached to the SageMaker notebook instance has full S3 access. What is the most likely cause?

Accepted Answer

The SageMaker execution role used for the training job does not have S3 access permissions.. The IAM role attached to the SageMaker notebook instance is used for interactive development, but training jobs run under a separate SageMaker execution role. Even if the notebook role has full S3 access, the training job's execution role must also have explicit S3 permissions. The 'AccessDenied' error indicates that the execution role lacks the necessary s3:GetObject or s3:ListBucket actions for the S3 bucket.

Answer

The S3 bucket has a bucket policy that denies access from the SageMaker service.

Answer

The training script is using an incorrect S3 bucket name.

Answer

The SageMaker training job is not configured to use the S3 VPC endpoint.

Question 10

An IAM policy is attached to a SageMaker execution role. A data scientist tries to create a training job using a custom algorithm stored in an ECR repository. The training job fails with an 'AccessDenied' error when pulling the Docker image from ECR. What is the missing permission?

Accepted Answer

ecr:GetDownloadUrlForLayer and ecr:BatchGetImage on the ECR repository. When SageMaker pulls a custom Docker image from ECR during training job creation, the execution role needs permissions to download the image layers. The required actions are ecr:GetDownloadUrlForLayer (to generate pre-signed URLs for each layer) and ecr:BatchGetImage (to retrieve image metadata and layer manifests). Without these, the 'AccessDenied' error occurs because SageMaker cannot authenticate or fetch the container image from the ECR repository.

Answer

ecr:PutImage on the ECR repository

Answer

s3:GetObject on the ECR repository

Answer

sagemaker:CreateTrainingJob on the ECR resource

Question 11

A DevOps engineer created a SageMaker notebook instance using the Terraform configuration shown. The notebook instance is in a VPC with a public subnet. However, the notebook instance cannot access the internet. What is the most likely cause?

Accepted Answer

The subnet does not have a route to an internet gateway.. A SageMaker notebook instance in a VPC with a public subnet requires a route to an internet gateway (IGW) in the subnet's route table to access the internet. Without that route, traffic from the notebook cannot reach the internet, even if `direct_internet_access` is enabled. The Terraform configuration likely omitted the route to the IGW, causing the connectivity failure.

Answer

The role_arn is incorrect or missing permissions.

Answer

The instance type ml.t2.medium does not support internet access.

Answer

The direct_internet_access parameter is set to 'Enabled' but should be 'Disabled'.

Question 12

A company is using Amazon SageMaker to train a XGBoost model on a large dataset. The training job is taking a long time. The data scientist wants to reduce training time without sacrificing model accuracy. The dataset is 100 GB in CSV format stored in S3. What is the most effective approach?

Accepted Answer

Use Pipe mode to stream data from S3 instead of downloading it first.. SageMaker's Pipe mode streams data directly from S3 to the training algorithm without writing it to disk, eliminating the I/O bottleneck of downloading the full 100 GB dataset. This reduces training time significantly by overlapping data loading with computation, while preserving model accuracy since the entire dataset is still used.

Answer

Reduce the number of instances to avoid communication overhead.

Answer

Use random sampling to reduce the dataset size to 10 GB.

Answer

Use SageMaker Managed Spot Training to reduce cost, but training time may increase due to interruptions.

Question 13

A data scientist is deploying a model using Amazon SageMaker for real-time inference. The model is memory-intensive and requires a GPU. Which instance type should be selected for the endpoint?

Accepted Answer

p3.2xlarge. The p3.2xlarge instance is correct because it provides a GPU (NVIDIA Tesla V100) with high memory bandwidth, which is essential for memory-intensive deep learning models requiring GPU acceleration for real-time inference. SageMaker endpoints for GPU-based models must use instance types from the P or G families, as CPU-only instances like i3, c5, or r5 lack the parallel processing capabilities needed for efficient GPU inference.

Answer

i3.2xlarge

Answer

c5.2xlarge

Answer

r5.2xlarge

Question 14

A data scientist is deploying a model on Amazon SageMaker for real-time inference. The model is a PyTorch model that requires custom inference code. The data scientist needs to handle variable-length inputs and optimize inference latency. Which TWO steps should the data scientist take? (Choose TWO.)

Accepted Answer

Use TorchScript to compile the model for optimized inference.. Option D is correct because TorchScript compiles the PyTorch model into a serialized, optimized representation that can be executed independently of Python, reducing per-request overhead and improving real-time inference latency, which directly addresses the optimization requirement. Option E is correct because SageMaker's PyTorch inference toolkit requires a custom inference.py script that implements model_fn to load the model and input_fn, predict_fn, and output_fn (or transform_fn) to handle variable-length inputs and custom preprocessing/postprocessing for real-time endpoints. Option A is incorrect because batch transform is an offline, asynchronous batch-processing feature, not a real-time inference endpoint, so it does not satisfy the real-time latency scenario. Option B is incorrect because an unmodified PyTorch container lacks the custom inference code needed to load the model and handle variable-length inputs. Option C is incorrect because multiple variants support A/B testing traffic splits, not latency optimization or variable-length input handling.

Answer

Enable SageMaker batch transform to process requests in batches.

Answer

Use the SageMaker PyTorch container without any modifications.

Answer

Set the endpoint to use multiple variants for A/B testing.

Question 15

A media company uses Amazon SageMaker to train a deep learning model for video classification. The training job uses a single ml.p3.2xlarge instance and processes 50 GB of labeled video data stored in Amazon S3. The training completes successfully in 12 hours. However, the data scientists report that the model’s accuracy is lower than expected. They suspect the training data contains labeling errors. To improve model accuracy without incurring significant additional cost, they want to identify and remove mislabeled training examples before retraining. They have a small budget of $50 and need to complete the analysis within 2 hours. Which approach should the data scientists take?

Accepted Answer

Train a small, fast model on a random sample of the data (e.g., 1 GB) using a cheaper instance like ml.m5.xlarge, then use the model's prediction confidence to flag low-confidence examples as potential mislabels for manual review.. Training a small, fast model on a 1 GB random sample using a cheaper instance (ml.m5.xlarge) allows the team to quickly identify low-confidence predictions, which are strong indicators of mislabeled examples. This approach fits within the $50 budget and 2-hour time constraint, as it avoids processing the full 50 GB dataset and leverages a lightweight model for rapid iteration. By flagging only suspicious samples for manual review, the team can efficiently clean the training data without incurring the cost of re-labeling the entire dataset.

Answer

Use SageMaker Ground Truth to create a new labeling job for the entire dataset, then compare the new labels with the original labels to identify discrepancies.

Answer

Use SageMaker Clarify to generate a bias report for the training data and remove instances that contribute to bias.

Answer

Manually review all 50 GB of video data to correct labels.