MLS-C01 Machine Learning Implementation and Operations Practice Test 29 — 15 Questions

Question 1

A data scientist is using Amazon SageMaker to train a model on a large dataset stored in Amazon S3. The training job is expected to run for several hours. The data scientist wants to reduce the cost of the training job without compromising model accuracy. The dataset is already in S3 in CSV format. Which two actions should the data scientist take to reduce training cost? (Choose two.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Enable SageMaker Pipe mode to stream data directly from S3 to the training container.. Managed Spot Training leverages discounted spare EC2 capacity, significantly lowering compute costs. Pipe mode reduces data download time and storage requirements, which can shorten training duration and enable the use of cheaper instances. Together, they provide direct cost savings without compromising model accuracy. Other options either have indirect or negative cost impacts.

Answer

Enable network isolation for the training job.

Answer

Convert the dataset to RecordIO format using the SageMaker XGBoost algorithm.

Answer

Use a larger instance type with more GPUs to finish training faster.

Question 2

A machine learning engineer is deploying a real-time inference endpoint on Amazon SageMaker. The model must handle occasional spikes in traffic and the engineer wants to minimize cost during periods of low traffic. The engineer configures automatic scaling for the endpoint using Application Auto Scaling. Which scaling policy configuration will best meet these requirements?

Accepted Answer

Target tracking scaling policy with a predefined metric of SageMakerVariantInvocationsPerInstance and a target value based on desired invocations per instance.. Target tracking with SageMakerVariantInvocationsPerInstance directly scales based on the number of invocations per instance, which reflects actual demand. This policy automatically increases instances during spikes and decreases them during low traffic, minimizing cost while maintaining performance. Other policies either require manual tuning, do not respond to real-time traffic, or use metrics that may not accurately represent load.

Answer

Step scaling policy that adds two instances when the CPU utilization exceeds 70% for 5 minutes and removes one instance when it drops below 30% for 10 minutes.

Answer

Scheduled scaling policy that scales the endpoint to a fixed number of instances during business hours and scales down to one instance at night.

Answer

Target tracking scaling policy with a predefined metric of CPUUtilization and a target value of 80%.

Question 3

A machine learning engineer is deploying a model to a SageMaker endpoint. The model requires a custom inference script that preprocesses input data. The engineer wants to minimize cold start latency and ensure the endpoint can handle sudden traffic spikes. Which deployment strategy should the engineer use?

Accepted Answer

Deploy the model to a real-time endpoint with a minimum instance count of 2 and auto scaling enabled.. Using a real-time endpoint with a minimum instance count of 2 and auto scaling ensures that at least two instances are always running, eliminating cold starts for those instances. Auto scaling quickly adds more instances when traffic increases, maintaining low latency. This strategy balances cost and performance for variable workloads.

Answer

Deploy the model using SageMaker Serverless Inference.

Answer

Deploy the model using SageMaker Asynchronous Inference.

Answer

Deploy the model to a real-time endpoint with a single instance and no auto scaling.

Question 4

A data scientist is using Amazon SageMaker to train a model on a large dataset stored in Amazon S3. The training job is expected to run for several hours. The data scientist wants to reduce the cost of the training job without compromising model accuracy. The dataset is already in S3 in CSV format. Which two actions should the data scientist take to reduce training cost? (Choose two.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Amazon CloudWatch Alarms with the CPUUtilization metric for the training job.. Amazon CloudWatch is the AWS service for monitoring metrics and setting alarms. SageMaker training jobs automatically publish metrics such as CPUUtilization to CloudWatch. By creating a CloudWatch Alarm on the CPUUtilization metric with a threshold and period, the data scientist can receive alerts when CPU usage exceeds 90% for 5 minutes. Other services like CloudTrail, Debugger, and Trusted Advisor do not provide this specific alerting capability.

Answer

AWS CloudTrail with a trail that logs SageMaker API calls and filters for high CPU events.

Answer

Amazon SageMaker Debugger with built-in rules to detect high CPU utilization.

Answer

AWS Trusted Advisor with a check for SageMaker training job CPU usage.

Question 5

A machine learning engineer has trained a SageMaker model and wants to deploy it for real-time inference. The model artifacts are stored in Amazon S3, and the engineer needs to ensure that the endpoint can automatically scale based on incoming traffic. The engineer also wants to minimize latency for the inference requests. Which deployment option should the engineer use?

Accepted Answer

SageMaker real-time endpoint with Application Auto Scaling. A real-time endpoint keeps the model loaded and ready, delivering low-latency predictions. Application Auto Scaling dynamically adjusts instance count based on CloudWatch metrics, providing automatic scaling. Together, they address both the latency and scaling needs. Other options either lack real-time capabilities or introduce delays unsuitable for low-latency requirements.

Answer

SageMaker asynchronous inference

Answer

SageMaker serverless inference

Answer

SageMaker batch transform

Question 6

A company has a SageMaker endpoint that serves a model for real-time predictions. The model was trained on data that includes categorical features. The company wants to monitor the endpoint for data drift and model quality. The company has enabled SageMaker Model Monitor with a baseline. After a week, the company notices that the data drift monitor is flagging violations, but the model's predictions still seem accurate. What is the most likely cause of the violations?

Accepted Answer

The baseline statistics were computed on a sample that is not representative of the live data distribution.. Data drift violations occur when live data deviates from the baseline. If the baseline was created from a sample that does not represent the live data distribution, the monitor will flag differences even though the model remains accurate. This is a common pitfall when the baseline is computed on a small or biased dataset. The other options would typically cause monitoring failures or actual performance degradation.

Answer

The endpoint is not sending data to the monitoring schedule because the IAM role lacks permissions.

Answer

The monitoring schedule is configured to monitor the wrong feature, such as a feature that is not present in the live data.

Answer

The model is suffering from concept drift, but the predictions are still accurate because the drift is in a feature that is not used by the model.

Question 7

A company has deployed a SageMaker endpoint for a fraud detection model. The model was trained on historical data and is performing well. However, the company notices that the model's predictions are becoming less accurate over time due to changes in customer behavior. The company wants to detect this degradation and receive alerts when the model's performance drops below a threshold. Which SageMaker feature should the company use?

Accepted Answer

SageMaker Model Monitor with a baseline and monitoring schedule. SageMaker Model Monitor is designed to continuously monitor the quality of predictions from a deployed model. By setting up a baseline and a monitoring schedule, it can detect data drift and model quality issues, and trigger CloudWatch alarms when violations exceed a threshold. This enables the company to be alerted when model performance degrades due to changing customer behavior.

Answer

SageMaker Clarify

Answer

SageMaker Inference Recommender

Answer

SageMaker Debugger with built-in rules

Question 8

A company is using Amazon SageMaker to train a deep learning model. The training job is taking longer than expected, and the data scientist suspects that the input data pipeline is the bottleneck. The training data consists of many small files stored in Amazon S3. The data scientist wants to improve training throughput by optimizing data loading. Which solution will MOST effectively reduce the training time?

Accepted Answer

Convert the small files into a single large file format such as TFRecord or RecordIO. Many small files create significant overhead due to per-file operations. Consolidating them into a few large files in an optimized format like TFRecord or RecordIO enables efficient sequential reads and better throughput. This directly addresses the data pipeline bottleneck, leading to faster training. Other options either do not target the bottleneck or may not provide the same level of improvement.

Answer

Enable SageMaker Managed Spot Training

Answer

Use Pipe mode instead of File mode in the SageMaker training job

Answer

Increase the number of training instances

Question 9

A machine learning engineer is deploying a model to a SageMaker endpoint and needs to monitor it for data drift and model quality. The engineer wants to receive alerts when the model's predictions deviate from expected behavior. Which two actions should the engineer take to set up automated monitoring and alerting? (Choose two.)

Accepted Answer

Create a CloudWatch alarm based on the Model Monitor metrics. SageMaker Model Monitor captures endpoint data and compares it to a baseline to detect drift and quality issues. It publishes metrics to CloudWatch, where alarms can be set to trigger notifications. Together, these provide automated monitoring and alerting. Other options do not directly monitor model performance or provide the required alerting.

Answer

Configure SageMaker Debugger to analyze training job tensors

Answer

Use AWS CloudTrail to log all API calls to the endpoint

Answer

Set up an AWS Lambda function to periodically invoke the endpoint and check responses

Question 10

A machine learning engineer needs to deploy a trained scikit-learn model to a SageMaker endpoint for real-time inference. The model artifacts are stored in Amazon S3. The engineer wants to use the SageMaker Python SDK to deploy the model with minimal code. Which method should the engineer use?

Accepted Answer

Use the SageMaker Python SDK's model.deploy() method with an instance type and initial instance count.. The SageMaker Python SDK's deploy() method is designed for deploying models to real-time endpoints with minimal code. It automatically creates the necessary endpoint configuration and endpoint, handling the deployment process efficiently. This is the most straightforward approach for the given scenario.

Answer

Use the SageMaker Python SDK's transformer.transform() method to create a real-time endpoint.

Answer

Create a model, endpoint configuration, and endpoint using the AWS CLI commands.

Answer

Use the SageMaker Python SDK's estimator.fit() method to train and deploy in one step.

Question 11

A machine learning engineer is tuning a SageMaker XGBoost model. They configure a hyperparameter tuning job with a maximum of 30 training jobs and set the objective metric as 'validation:auc'. After the tuning job completes, they want to deploy the model that achieved the best validation AUC. Which step will identify the best training job?

Accepted Answer

Call DescribeHyperParameterTuningJob and use the BestTrainingJob field.. After a hyperparameter tuning job completes, the BestTrainingJob field in DescribeHyperParameterTuningJob provides the name of the training job that achieved the best objective metric. Deploying that model ensures you use the optimal hyperparameters found during tuning. Other methods do not reliably identify the best model.

Answer

Use the SageMaker console to sort the tuning job's training jobs by creation date and pick the latest.

Answer

List all training jobs with ListTrainingJobs and select the one with the shortest training time.

Answer

Retrieve the best model from the tuning job's S3 output path by reading the manifest file.

Question 12

A company runs a SageMaker real-time endpoint that serves a fraud detection model. The security team requires that all inference requests and responses be recorded for audit, and the data science team wants to detect feature drift in the incoming request data over time. The endpoint currently has no monitoring configured. Which two actions should the engineer take to meet both requirements? (Choose two.)

Accepted Answer

Create a Model Monitor data quality monitoring schedule that baselines the training data and compares captured data against it.. Meeting both goals requires recording production traffic and then analyzing it. Data capture writes inference requests and responses to Amazon S3, creating the audit record and the raw input for analysis. A Model Monitor data quality schedule compares that captured data against a baseline built from the training set and raises violations when feature distributions shift, which is how drift is detected. Permissions, scaling, and Debugger do not produce either artifact.

Answer

Enable SageMaker Debugger on the endpoint to capture tensor values from the deployed model.

Answer

Attach an IAM policy to the endpoint's execution role granting s3:PutObject on the captured data prefix.

Answer

Configure the endpoint's auto scaling policy to trigger on the number of model invocations per instance.

Question 13

A data scientist is using Amazon SageMaker to train a model and wants to ensure that the training job can access the training data stored in an Amazon S3 bucket. The S3 bucket is in the same AWS account and Region. What is the SIMPLEST way to grant the necessary permissions?

Accepted Answer

Create an IAM role with a policy that allows s3:GetObject on the bucket and pass it as the execution role to the training job. SageMaker training jobs use an IAM execution role to access AWS resources. Creating a role with the necessary S3 read permissions and specifying it when launching the training job is the simplest and most secure method. Other options either do not apply to the training job's context or pose security risks.

Answer

Make the S3 bucket public so that the training job can access the data without credentials

Answer

Create an IAM role with the AmazonS3ReadOnlyAccess policy and attach it to the SageMaker notebook instance

Answer

Use the AWS root user credentials to create the training job

Question 14

A company is using Amazon SageMaker to run a hyperparameter tuning job for an XGBoost model. The tuning job must efficiently search a large hyperparameter space while minimizing training time and cost. Which two strategies should the company use to achieve these goals? (Choose two.)

Accepted Answer

Enable early stopping in the hyperparameter tuning job to stop poorly performing training jobs before they complete.. Bayesian optimization is sample-efficient and finds good hyperparameters with fewer trials, reducing cost and time. Early stopping terminates underperforming trials early, further saving resources. Together, they efficiently search a large space while minimizing training time. Grid search and excessive job counts increase cost, while random search with too few jobs may not find optimal hyperparameters.

Answer

Set the maximum number of training jobs to a very high value to ensure thorough exploration.

Answer

Use grid search to exhaustively evaluate all possible hyperparameter combinations.

Answer

Use random search with a small number of training jobs to quickly sample the hyperparameter space.

Question 15

A data scientist is preparing a training script that will run in script mode on Amazon SageMaker. The script must be able to read the training dataset that SageMaker copies from Amazon S3, and the data scientist wants to avoid hard-coding any bucket names or local paths. Which approach should the data scientist use?

Accepted Answer

Read the SM_CHANNEL_TRAIN environment variable inside the script to obtain the local directory containing the training channel data.. SageMaker's training toolkit stages each input channel from Amazon S3 onto local storage in the container and publishes the resulting directory through SM_CHANNEL_<CHANNEL_NAME> environment variables. Reading SM_CHANNEL_TRAIN makes the script portable across buckets, account environments, and pipeline runs because the path is injected at runtime. Hyperparameter and resource configuration files serve different purposes and do not carry channel paths.

Answer

Call the SageMaker ListTrainingJobs API from inside the script and parse the InputDataConfig to find the S3 URI.

Answer

Read the training data from the /opt/ml/input/config/hyperparameters.json file, which contains the dataset location.

Answer

Hard-code the S3 URI of the training data and use boto3 to download it at the start of the script.