MLS-C01 Machine Learning Implementation and Operations Practice Test 10 — 15 Questions

Question 1

A company is using Amazon SageMaker Ground Truth to create labeled datasets for a text classification task. The labeling job uses a private workforce of 10 annotators. After labeling 10,000 items, the quality of labels is inconsistent. Which approach will MOST effectively improve labeling consistency?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Configure the labeling job to use annotation consolidation with majority voting and require multiple annotations per item.. Requiring multiple annotations per item and using annotation consolidation with majority voting directly improves labeling consistency by reducing individual annotator bias and ensuring that the final label is based on consensus. Option A is incorrect because removing annotations from low-agreement annotators after the job completes does not improve the quality of labels already assigned; it may also discard valid data. Option B is incorrect because simply increasing the number of annotators does not guarantee consistency; it may introduce more variance without a consolidation mechanism. Option D is incorrect because active learning is used to select which items to label, not to improve the consistency of labeling; it does not address the inconsistency in the labeling process itself.

Answer

Remove annotations from annotators with low agreement after the job completes.

Answer

Increase the number of annotators to 20 to average out inconsistencies.

Answer

Use active learning to automatically label the most confident samples and only send uncertain ones to annotators.

Question 2

A data scientist trains a model using Amazon SageMaker's built-in XGBoost algorithm. The model overfits on the training data. Which hyperparameter adjustment is MOST likely to reduce overfitting?

Accepted Answer

Increase the value of the lambda (L2 regularization) hyperparameter.. Increasing the lambda (L2 regularization) hyperparameter adds a penalty on the squared magnitude of the model weights, which discourages the model from fitting noise in the training data. This directly reduces overfitting by shrinking the influence of individual features, a standard regularization technique in XGBoost.

Answer

Increase the value of the max_depth hyperparameter.

Answer

Increase the value of the subsample hyperparameter to 1.0.

Answer

Increase the value of the num_round hyperparameter.

Question 3

A company is building a recommendation system using Amazon SageMaker. The training data includes user-item interactions stored in a DataFrame with over 100 million rows. The data scientist wants to perform feature engineering, including one-hot encoding of categorical features with high cardinality. Which approach is MOST cost-effective and scalable?

Accepted Answer

Use SageMaker Processing with a Spark container to distribute the encoding job.. SageMaker Processing with a Spark container allows distributed execution of one-hot encoding on high-cardinality categorical features across a managed cluster, scaling horizontally to handle over 100 million rows without manual infrastructure management. This approach is cost-effective as you pay only for the processing time, and it integrates natively with SageMaker for seamless data pipeline orchestration.

Answer

Use Amazon EMR with Spark and store the processed data in HDFS.

Answer

Use a SageMaker notebook instance with scikit-learn to perform the encoding in memory.

Answer

Use AWS Glue ETL jobs to perform the encoding and store the result in S3.

Question 4

A machine learning team is using Amazon SageMaker Experiments to track multiple training runs. They need to compare the performance of different models based on metrics like accuracy and F1 score. However, when they view the experiment list in SageMaker Studio, the metrics are not displayed. What is the MOST likely cause?

Accepted Answer

The training script did not use the SageMaker SDK to log the metrics.. SageMaker Experiments automatically tracks parameters and metrics only when the training script explicitly logs them using the SageMaker SDK's `log_metric` function or the `sagemaker.experiments.run.Run` class. Without these SDK calls, the metrics are never recorded in the experiment's trial components, so they will not appear in the Studio experiment list. The team must instrument their training code to emit metrics for Experiments to capture them.

Answer

The training job did not define metric definitions in the algorithm specification.

Answer

The training job is running on an instance type that does not support Experiments.

Answer

The IAM role used by SageMaker does not have permission to write to the Experiments table.

Question 5

A company is deploying a PyTorch model on a SageMaker endpoint for real-time inference. The model is stored as a .pth file in an S3 bucket. The data scientist wants to use the SageMaker PyTorch inference toolkit. Which file is REQUIRED in the model artifacts to serve the model?

Accepted Answer

A file named model.pth containing the model state dictionary.. The SageMaker PyTorch inference toolkit expects the model artifact to be a single file named model.pth containing the model's state dictionary. When using the default inference handler, the toolkit automatically loads this file into the PyTorch model for serving. No additional inference script is required if the default behavior is sufficient.

Answer

A file named model.tar.gz that contains the model and any dependencies.

Answer

A file named inference.py that defines the model loading and prediction logic.

Answer

A file named requirements.txt listing the dependencies.

Question 6

A data scientist is using Amazon SageMaker Autopilot to automatically build a binary classification model. After the Autopilot job completes, the best model has an accuracy of 0.85 on the validation set. However, the data scientist notices a class imbalance (90% negative, 10% positive). Which metric should the data scientist use to evaluate the model's performance on the positive class?

Accepted Answer

Area Under the ROC Curve (AUC). A is correct because Area Under the ROC Curve (AUC) is threshold-independent and evaluates the model's ability to distinguish between positive and negative classes across all classification thresholds. In the presence of severe class imbalance (90% negative, 10% positive), AUC provides a robust measure of model performance on the positive class without being skewed by the majority class, unlike accuracy which would be high even if the model predicts all negatives.

Answer

Recall

Answer

Accuracy

Answer

Precision

Question 7

A company is using Amazon SageMaker to train a deep learning model for image classification. The training job is using a single p3.2xlarge instance and takes 10 hours. The data scientist wants to reduce training time using distributed training. Which SageMaker feature should be used?

Accepted Answer

Use the SageMaker distributed data parallelism library with multiple p3.2xlarge instances.. The goal is to reduce training time, and the SageMaker distributed data parallelism library is designed to split the mini-batch across multiple GPU instances, enabling synchronous or asynchronous gradient updates that scale near-linearly with the number of instances. By adding more p3.2xlarge instances, the effective throughput increases, directly reducing wall-clock training time for the image classification model.

Answer

Use SageMaker Managed Spot Training to reduce cost, but training time remains the same.

Answer

Use SageMaker Hyperparameter Tuning to find optimal hyperparameters faster.

Answer

Use the SageMaker distributed model parallelism library with a single p3dn.24xlarge instance.

Question 8

A company is deploying a machine learning model using Amazon SageMaker. The model needs to be updated frequently with new data. Which TWO approaches can be used to update the model without downtime? (Choose TWO.)

Accepted Answer

Use SageMaker A/B testing to gradually shift traffic to the new model variant.. Amazon SageMaker supports deploying multiple model variants behind a single endpoint using production variants. By using A/B testing (traffic shifting), you can gradually route a percentage of inference requests to the new model variant while the old variant continues serving the majority of traffic, enabling updates with zero downtime.

Answer

Delete the existing endpoint and create a new one with the updated model.

Answer

Directly update the model artifact in the existing endpoint configuration.

Answer

Stop the endpoint, update the model, and restart the endpoint.

Question 9

A data scientist is using Amazon SageMaker to train a model using a custom Docker container. The training job fails with an error message indicating that the container exited with a non-zero code. Which THREE steps should the data scientist take to diagnose the issue? (Choose THREE.)

Accepted Answer

Use the SageMaker Debugger to capture system metrics and output tensors for analysis.. SageMaker Debugger can capture system metrics (e.g., CPU/GPU utilization, memory) and output tensors during training, which helps identify issues like resource exhaustion or problematic gradients that cause non-zero exit codes. It provides deep visibility into the training process without requiring code changes, making it a powerful diagnostic tool for custom container failures.

Answer

Retry the training job with the same configuration; the error might be transient.

Answer

Increase the number of training instances to distribute the workload.

Question 10

A company is building a machine learning pipeline on AWS. The pipeline includes data ingestion, preprocessing, training, and deployment. Which THREE AWS services can be used to orchestrate the pipeline? (Choose THREE.)

Accepted Answer

AWS Glue Workflows. AWS Glue Workflows is correct because it provides a visual orchestration tool for designing and managing complex ETL pipelines, including data ingestion and preprocessing steps. It allows you to define dependencies, triggers, and job sequences, making it suitable for orchestrating the data preparation stages of a machine learning pipeline.

Answer

Amazon CloudWatch

Answer

AWS Lambda

Question 11

A data scientist wants to deploy a PyTorch model for real-time inference with low latency. Which AWS service should they use?

Accepted Answer

Amazon SageMaker real-time endpoint. Amazon SageMaker real-time endpoints are specifically designed for low-latency inference on deployed models, including PyTorch models. They provide persistent HTTPS endpoints that autoscale and support custom containers, making them ideal for real-time prediction workloads.

Answer

Amazon Elastic Container Service (ECS)

Answer

Amazon SageMaker batch transform

Answer

AWS Lambda

Question 12

A company's ML model training on Amazon SageMaker is taking longer than expected. The training job uses a single ml.p3.2xlarge instance. Which change is most likely to reduce training time?

Accepted Answer

Use distributed training with multiple GPU instances. The training job is bottlenecked by compute capacity, as a single ml.p3.2xlarge instance provides only one NVIDIA V100 GPU. Distributed training with multiple GPU instances (e.g., multiple ml.p3.2xlarge instances) enables data parallelism, splitting the workload across GPUs and significantly reducing wall-clock training time for large models or datasets.

Answer

Increase the instance's EBS volume size

Answer

Enable Managed Spot Training

Answer

Switch to a compute-optimized instance with more vCPUs

Question 13

A team is using Amazon SageMaker Autopilot to automatically build models. The dataset has 50 features and 1 million rows. After training, Autopilot generates multiple candidates. The team wants to deploy the model with the highest accuracy. What is the best practice to select and deploy the model?

Accepted Answer

Deploy the candidate with the highest objective metric value from the Autopilot leaderboard. Amazon SageMaker Autopilot automatically generates a leaderboard ranking candidate models by their objective metric (e.g., accuracy, F1, AUC). The highest-ranked candidate represents the best-performing model based on the validation data, and deploying it directly is the recommended best practice. Autopilot handles preprocessing, algorithm selection, and hyperparameter tuning internally, so manual intervention is unnecessary.

Answer

Deploy all candidates behind a multi-model endpoint and route traffic based on request features

Answer

Select the model with the highest validation accuracy after performing additional hyperparameter tuning

Answer

Manually review each candidate's architecture and select the one with the simplest design

Question 14

An ML engineer needs to store and version training datasets and model artifacts. Which AWS service should they use?

Accepted Answer

Amazon Simple Storage Service (S3). Amazon S3 is the correct choice because it provides scalable, durable, and cost-effective object storage with built-in versioning capabilities, making it ideal for storing and versioning large training datasets and model artifacts. S3's versioning feature allows you to preserve, retrieve, and restore every version of an object, which is essential for reproducibility in ML workflows.

Answer

Amazon DynamoDB

Answer

Amazon Elastic File System (EFS)

Answer

Amazon Elastic Block Store (EBS)

Question 15

A team is training a large language model using PyTorch on multiple GPUs. The training is taking too long due to inefficient data loading. Which AWS service can help accelerate data loading by caching data close to the GPU instances?

Accepted Answer

Amazon FSx for Lustre. Amazon FSx for Lustre is a high-performance file system optimized for machine learning workloads. It provides sub-millisecond latencies and high throughput by caching training data on local NVMe SSDs attached to the Lustre servers, which are co-located with GPU instances in the same AWS Availability Zone. This eliminates the I/O bottleneck from remote object storage, directly accelerating data loading for PyTorch DataLoader workers.

Answer

Amazon EBS Snapshots for fast restore

Answer

Amazon S3 Transfer Acceleration

Answer

Amazon CloudFront