MLS-C01 Modeling Practice Test 16 — 15 Questions

Question 1

A healthcare company is building a model to predict patient readmission within 30 days. They have structured electronic health records (EHR) data with 200 features. The data includes missing values, categorical variables with high cardinality (e.g., diagnosis codes), and a severe class imbalance (5% readmission). They need to deploy a model on SageMaker that is interpretable and achieves high recall for the positive class. Which combination of techniques should they use?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use XGBoost with SMOTE, feature selection via SHAP, and deploy as a SageMaker endpoint. XGBoost natively handles missing values, making it well-suited for EHR data with missing entries. SMOTE addresses the severe class imbalance by generating synthetic samples of the minority class, which improves recall. SHAP feature selection provides interpretability by identifying the most influential features, and deploying as a SageMaker endpoint enables real-time predictions. This combination directly meets the requirements of high recall and interpretability.

Answer

Use logistic regression with one-hot encoding and random undersampling

Answer

Use PCA for dimensionality reduction, then train a linear SVM with class weights

Answer

Use a deep neural network with embeddings for categorical variables and oversample the minority class

Question 2

A company uses SageMaker to deploy a real-time inference endpoint for a fraud detection model. The model is an XGBoost model trained on 50 features. The endpoint receives 100 requests per second, but latency is higher than the required 200 ms. The team wants to reduce latency without retraining. What should they do?

Accepted Answer

Reduce the number of features by selecting the most important ones. To reduce inference latency without retraining the XGBoost model, reducing the number of features to the most important ones directly decreases the computational complexity of the model, as fewer tree splits are evaluated per request. This is a model-level optimization that does not require retraining if the feature importance is already known. SageMaker Elastic Inference, however, is designed to accelerate deep learning models by attaching a GPU accelerator; it does not speed up XGBoost or other tree-based models because they do not utilize GPUs effectively. Therefore, only option C is correct.

Answer

Increase the number of instances behind the endpoint

Answer

Use SageMaker's batch transform instead of real-time endpoint

Answer

Use SageMaker's Elastic Inference to attach an acceleration to the endpoint

Question 3

A company is using SageMaker's built-in image classification algorithm to classify product images into 100 categories. The training takes 3 hours on a single p3.2xlarge instance. They need to reduce training time to under 1 hour. They have access to a cluster of 4 p3.2xlarge instances. Which approach should they take?

Accepted Answer

Use SageMaker's distributed training with data parallelism using Horovod. SageMaker's built-in image classification algorithm supports distributed training with data parallelism using Horovod, which splits the mini-batch across multiple GPUs and synchronizes gradients via allreduce. With 4 p3.2xlarge instances (each with 1 GPU), this reduces per-iteration time proportionally, enabling the 3-hour job to complete in under 1 hour when scaling batch size and learning rate appropriately.

Answer

Use SageMaker's hyperparameter tuning to find faster convergence

Answer

Use a smaller batch size on each instance

Answer

Use SageMaker's managed spot training with checkpointing

Question 4

A bank is building a credit risk model using a large dataset with 500 features and 2 million samples. The dataset contains many categorical features with high cardinality (e.g., zip code, occupation). The model must be deployed on SageMaker and provide real-time predictions with low latency. They also need to explain individual predictions for regulatory compliance. Which approach is most appropriate?

Accepted Answer

Use a gradient boosting model with ordinal encoding for categorical features and use SageMaker's built-in XGBoost with SHAP. XGBoost with ordinal encoding and SHAP balances performance, latency, and explainability.

Answer

Use a linear model with target encoding for categorical features and deploy with SageMaker's built-in linear learner algorithm

Answer

Use a deep neural network with embedding layers for categorical features and use SageMaker's built-in Debugger for explanations

Answer

Use XGBoost with one-hot encoding for categorical features and deploy with SageMaker's built-in SHAP explainer

Question 5

A company uses Amazon SageMaker to train a linear regression model on a dataset with 10 million rows and 50 features. The training job takes 8 hours to complete. A data scientist wants to reduce the training time to under 2 hours without changing the dataset size or the model algorithm. The SageMaker instance type currently used is ml.m5.2xlarge. Which action should the data scientist take to achieve the desired training time?

Accepted Answer

Use SageMaker's distributed training with 4 ml.m5.2xlarge instances.. Distributed training with 4 ml.m5.2xlarge instances allows the training job to parallelize across multiple instances, reducing the overall training time. Since the dataset and algorithm remain unchanged, using distributed training is the most effective way to achieve the desired speedup without altering the data or model. This approach can scale nearly linearly if the algorithm supports data parallelism.

Answer

Change the instance type to ml.p3.2xlarge (GPU instance).

Answer

Change the instance type to ml.m5.4xlarge (double the vCPUs and memory).

Answer

Reduce the number of features from 50 to 25.

Question 6

A machine learning team is deploying a real-time inference endpoint for a fraud detection model using Amazon SageMaker. The model is a LightGBM classifier trained on 1 GB of tabular data. The endpoint must respond within 100 ms for 99% of requests, with a throughput of 10 requests per second. During load testing, the team observes that the 99th percentile latency is 250 ms and the endpoint CPU utilization is consistently above 90%. The team has already selected an ml.c5.xlarge instance with auto scaling enabled. Which combination of actions should the team take to meet the latency requirement? (Choose 3.)

Accepted Answer

Upgrade the instance type to ml.c5.2xlarge to increase CPU resources per instance.. Option A is correct because the endpoint is CPU-bound (CPU utilization consistently above 90%), so moving from ml.c5.xlarge to ml.c5.2xlarge doubles the vCPU and memory per instance, giving each inference more compute headroom to bring the 99th percentile latency down toward 100 ms. Option B is correct because LightGBM inference time scales with the number of trees and their depth; pruning or reducing the tree count directly lowers per-request compute, which is the bottleneck causing the 250 ms p99 latency. Option C is correct because enabling payload compression (e.g., gzip) reduces the bytes transferred over the network and the deserialization work on the endpoint, cutting the network/overhead portion of the request latency for the 10 RPS workload. Option D is not appropriate because SageMaker Batch Transform is an asynchronous, job-based service for offline scoring of large datasets and cannot satisfy a real-time 100 ms p99 latency requirement for an interactive fraud detection endpoint.

Answer

Switch to using SageMaker Batch Transform instead of a real-time endpoint.

Question 7

A data scientist is using Amazon SageMaker to train a neural network for a regression task. The training loss decreases but the validation loss starts to increase after several epochs. The data scientist wants to automatically stop training when validation loss no longer improves, without manual intervention. Which SageMaker feature should be used?

Accepted Answer

Configure SageMaker Early Stopping in the training job's algorithm-specific parameters.. Early stopping is the standard technique to halt training when validation performance degrades. SageMaker provides built-in early stopping for several algorithms and allows custom implementations via callbacks. Debugger is for monitoring, AMT is for hyperparameter search, and fixed epochs are static. Thus, configuring early stopping is the correct approach.

Answer

Set the 'max_epochs' hyperparameter to a low value based on prior experiments.

Answer

Enable SageMaker Automatic Model Tuning (AMT) with a hyperparameter tuning job.

Answer

Use SageMaker Debugger to monitor validation loss and manually stop the job.

Question 8

A data scientist is preparing a dataset for training a linear regression model using Amazon SageMaker. The dataset contains a categorical feature 'city' with 50 unique values. The data scientist wants to encode this feature appropriately for the model. Which two steps should be taken? (Choose two.)

Accepted Answer

Drop one of the one-hot encoded columns to avoid multicollinearity (dummy variable trap).. For linear regression, categorical features must be encoded numerically. One-hot encoding is appropriate to avoid ordinal assumptions, and dropping one dummy column prevents multicollinearity. Label encoding and target encoding are not ideal here, and PCA is for numerical dimensionality reduction.

Answer

Use the SageMaker built-in PCA algorithm to reduce the dimensionality of the 'city' feature.

Answer

Apply target encoding, replacing each city with the mean of the target variable for that city.

Answer

Use label encoding to assign each city a unique integer from 1 to 50.

Question 9

A machine learning engineer is training a linear regression model on a dataset with 50 features using Amazon SageMaker's built-in Linear Learner algorithm. The training loss decreases initially but then starts to oscillate significantly and the model performs poorly on the validation set. The engineer suspects overfitting. Which of the following techniques is specifically designed within the Linear Learner algorithm to address this issue?

Accepted Answer

Use the 'early_stopping' hyperparameter to stop training when validation loss stops improving.. The Linear Learner algorithm includes an early stopping feature that can be enabled via the 'early_stopping' hyperparameter. When validation loss fails to improve over a specified number of epochs, training halts, mitigating overfitting. This is a built-in mechanism specifically designed for this purpose, making it the correct choice.

Answer

Enable L1 regularization by setting the 'l1' hyperparameter.

Answer

Increase the number of epochs.

Answer

Set the 'use_bias' hyperparameter to false.

Question 10

A machine learning engineer is using Amazon SageMaker Automatic Model Tuning to optimize a deep learning model. The tuning job has completed 50 training jobs, but the best objective metric has not improved in the last 15 jobs. The engineer wants to stop the tuning job early to save cost while preserving the best model found so far. Which action should the engineer take?

Accepted Answer

Enable early stopping in the tuning job configuration and set the max_number_of_training_jobs to a value larger than 50.. SageMaker Automatic Model Tuning supports early stopping to terminate a tuning job when the objective metric stops improving. By enabling early stopping and setting a high max_number_of_training_jobs, the engineer allows the early stopping mechanism to halt the job once the specified patience is exceeded, while ensuring the job count limit does not interfere. The best model from all completed jobs is preserved and can be deployed.

Answer

Configure the tuning job with early stopping and set the max_number_of_training_jobs to 50.

Answer

Stop the tuning job manually and deploy the best model from the completed training jobs.

Answer

Increase the value of the max_parallel_jobs parameter to speed up the remaining training jobs.

Question 11

A machine learning engineer is using Amazon SageMaker to train a model that predicts customer churn. The dataset contains a categorical feature "region" with 50 unique values. The engineer wants to avoid imposing an artificial ordinal relationship and also keep the feature dimension manageable. Which encoding approach should be used?

Accepted Answer

Embedding layer that learns a dense representation for each category.. An embedding layer learns a dense, low-dimensional representation for each category, which avoids imposing an artificial order and reduces dimensionality compared to one-hot encoding. It is particularly effective for high-cardinality categorical features like region with 50 values. The other methods either impose ordinality, increase dimensionality excessively, or risk target leakage.

Answer

Target encoding using the mean of the target variable for each category.

Answer

One-hot encoding with all 50 categories.

Answer

Label encoding with arbitrary integer assignments.

Question 12

A data scientist is training a gradient boosting model using the SageMaker built-in XGBoost algorithm on a tabular dataset with a highly skewed target variable. The model performs poorly on the minority class in validation. The scientist wants to improve performance without collecting new data. After enabling scale_pos_weight in the hyperparameters, which additional SageMaker built-in XGBoost hyperparameter should be tuned to directly control the trade-off between training loss and model complexity?

Accepted Answer

lambda. In the SageMaker built-in XGBoost algorithm, the lambda hyperparameter is the L2 regularization term on leaf weights. It directly governs the trade-off between minimizing training loss and keeping the model simple. When dealing with skewed targets, combining scale_pos_weight with a well-tuned lambda can reduce overfitting to the majority class and improve minority-class recall. Other hyperparameters like max_depth or subsample affect complexity indirectly but do not explicitly parameterize this trade-off.

Answer

max_depth

Answer

alpha

Answer

subsample

Question 13

A data scientist is training a model using Amazon SageMaker and wants to evaluate its performance on a held-out test set. The dataset is stored in Amazon S3 and is already split into training and test sets. Which SageMaker feature should be used to compute evaluation metrics after training?

Accepted Answer

SageMaker Batch Transform. To evaluate a trained model on a held-out test set, the scientist needs to generate predictions. Batch Transform does this efficiently for large datasets without needing an endpoint. Automatic Model Tuning, Debugger, and Model Monitor serve different purposes: hyperparameter search, training diagnostics, and production monitoring, respectively.

Answer

SageMaker Debugger

Answer

SageMaker Model Monitor

Answer

SageMaker Automatic Model Tuning

Question 14

A machine learning engineer is deploying a model to an Amazon SageMaker endpoint for real-time inference. The model is a deep neural network that requires GPU acceleration and must handle a high volume of requests with low latency. The engineer wants to optimize the endpoint for cost and performance. Which deployment strategy should the engineer use?

Accepted Answer

Use an Amazon SageMaker endpoint with an ml.p3.2xlarge instance and enable automatic scaling.. For a deep neural network requiring GPU acceleration and high throughput with low latency, a GPU instance like ml.p3.2xlarge combined with automatic scaling provides the necessary compute and elasticity. Elastic Inference is for CPU instances, multi-model endpoints are for hosting many models, and serverless lacks GPU support.

Answer

Deploy the model to a multi-model endpoint on a CPU instance.

Answer

Use a serverless inference endpoint with a memory size of 6 GB.

Answer

Use a single ml.p3.2xlarge instance with SageMaker Elastic Inference.

Question 15

A financial institution is building a model to predict loan defaults using Amazon SageMaker. The dataset is highly imbalanced, with only 2% of loans defaulting. The team wants to evaluate the model's ability to identify defaulters while controlling the false positive rate. Which metric should the team use to optimize the model?

Accepted Answer

Area Under the Receiver Operating Characteristic Curve (AUC-ROC). AUC-ROC is the best choice because it evaluates the model's discriminative ability across all thresholds, balancing true positive and false positive rates. This is crucial for imbalanced datasets where accuracy is misleading. Other metrics like precision or recall focus on only one aspect, while AUC-ROC provides a holistic view, aligning with the need to identify defaulters while controlling false positives.

Answer

Precision

Answer

Accuracy

Answer

Recall