MLS-C01 Modeling Practice Test 49 — 15 Questions

Question 1

A machine learning engineer is using Amazon SageMaker to train a deep learning model for image segmentation using the built-in Semantic Segmentation algorithm. The training dataset consists of 10,000 images with pixel-level annotations. The engineer notices that the model's validation loss is much higher than the training loss after 50 epochs, and the model performs poorly on a held-out test set. Which SageMaker training configuration change is MOST likely to improve the model's generalization?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Add data augmentation to the training pipeline, such as random cropping and flipping.. The model is overfitting, as indicated by high validation loss compared to training loss. Data augmentation is a powerful regularization technique that expands the effective training set with label-preserving transformations. For image segmentation, random cropping and flipping help the model generalize to variations in object position and orientation. Early stopping only stops training but doesn't improve the model's ability to generalize; increasing epochs or reducing learning rate do not address the root cause.

Answer

Reduce the learning rate to make the training more stable.

Answer

Increase the number of epochs to 100 to allow the model to learn more from the training data.

Answer

Enable early stopping with a patience parameter based on validation loss.

Question 2

A machine learning engineer is training a gradient boosting model on Amazon SageMaker using the built-in XGBoost algorithm. The dataset contains a highly imbalanced binary target (2% positive). The engineer wants to improve recall without collecting more data. Which SageMaker XGBoost hyperparameter should be adjusted to directly address class imbalance during training?

Accepted Answer

scale_pos_weight. The SageMaker built-in XGBoost algorithm exposes scale_pos_weight specifically to handle skewed binary classification by weighting positive-class gradients. Setting it near the negative-to-positive ratio makes the boosting process pay proportionally more attention to the minority class, which raises recall. Other hyperparameters such as tree count, learning rate, and depth affect capacity and optimization speed but leave the class-wise loss weighting unchanged, so they cannot directly correct the imbalance.

Answer

max_depth

Answer

eta

Answer

num_round

Question 3

A fraud-detection team at a bank trains an XGBoost binary classifier on 2 million transactions where only 0.8% are fraudulent. The model achieves 0.995 AUC but the fraud team reports that at the default 0.5 probability threshold, recall for the fraud class is 0.31, meaning most frauds are missed. The team wants to increase fraud recall without retraining the model. Which approach should they take?

Accepted Answer

Adjust the classification threshold used at inference time below 0.5 to trade precision for higher recall on the minority class.. When a classifier ranks well but operates at an inappropriate cut point, the operating point is the problem, not the model. For rare-event detection, the default 0.5 threshold is rarely aligned with the business cost of false negatives. Adjusting the threshold at inference time raises minority-class recall using the existing model, satisfying the constraint of no retraining.

Answer

Increase the number of boosting rounds until the validation AUC exceeds 0.999, then redeploy the model to the endpoint.

Answer

Retrain the model with the scale_pos_weight hyperparameter set to a value near 1.0 so the two classes are weighted equally.

Answer

Apply SMOTE to the training data and retrain so the positive class represents roughly 50% of the samples.

Question 4

A data scientist is using Amazon SageMaker to train a deep learning model for image segmentation. The model must output a mask for each pixel, and the training dataset is highly imbalanced with many background pixels and few foreground pixels. Which loss function should the data scientist use to address the class imbalance and improve segmentation performance?

Accepted Answer

Dice loss. Dice loss is specifically designed for segmentation tasks and is robust to class imbalance because it measures the overlap between predicted and true masks. It effectively ignores the large number of true negatives and focuses on the foreground. In contrast, binary cross-entropy can be overwhelmed by background pixels, while MSE and hinge loss are not suitable for segmentation.

Answer

Mean squared error loss

Answer

Binary cross-entropy loss

Answer

Hinge loss

Question 5

A data scientist is using Amazon SageMaker to train a linear learner model on a dataset with 100 features. After training, the model's weights are inspected, and many are very close to zero. The data scientist suspects that irrelevant features are affecting the model's performance. Which SageMaker Linear Learner hyperparameter should be adjusted to perform feature selection and improve model performance?

Accepted Answer

Set the 'l1' hyperparameter to a non-zero value.. L1 regularization, controlled by the 'l1' hyperparameter in SageMaker Linear Learner, induces sparsity by driving some coefficients to zero. This effectively selects a subset of features, which is beneficial when many features are irrelevant. L2 regularization only shrinks coefficients and does not perform feature selection. Adjusting learning rate or bias does not address feature relevance.

Answer

Set the 'l2' hyperparameter to a non-zero value.

Answer

Increase the 'learning_rate' to speed up convergence.

Answer

Set the 'use_bias' hyperparameter to false.

Question 6

A machine learning engineer is training a linear regression model on a dataset with 20 features using Amazon SageMaker's built-in Linear Learner algorithm. The training job completes quickly, but during validation, the model's predictions are consistently poor, with high mean squared error. The engineer suspects that the features have vastly different scales and distributions. Which preprocessing step should be applied to the training data to most likely improve the model's performance?

Accepted Answer

Standardize the features by subtracting the mean and dividing by the standard deviation.. The scenario describes a linear model struggling with features on different scales, a common problem that leads to slow convergence and poor performance. Standardizing features to zero mean and unit variance ensures that all features contribute equally to the model's learning process. This preprocessing step is a standard recommendation for linear models and directly addresses the suspected cause of high error.

Answer

Apply one-hot encoding to all numerical features.

Answer

Increase the number of epochs for training.

Answer

Remove outliers from the dataset using a z-score threshold.

Question 7

A team is training a SageMaker built-in Linear Learner model for a regression task predicting house prices. The training data contains a few extreme outliers in the target variable. The team wants the model to be robust to these outliers while still using the built-in algorithm. Which predictor_type and loss combination should they configure?

Accepted Answer

predictor_type='regressor' with loss='absolute_loss'. SageMaker Linear Learner supports multiple losses per predictor type. For regression with outlier-prone targets, absolute loss reduces the influence of large residuals because its gradient is bounded in magnitude, unlike squared loss whose gradient scales with the residual. Keeping predictor_type as regressor preserves the continuous output. Classification modes are structurally incapable of predicting prices, so the loss choice within regressor mode is the deciding factor.

Answer

predictor_type='multiclass_classifier' with loss='softmax'

Answer

predictor_type='binary_classifier' with loss='logistic'

Answer

predictor_type='regressor' with loss='squared_loss'

Question 8

A machine learning engineer is using Amazon SageMaker to train a model with the built-in Linear Learner algorithm. The dataset has a large number of features, and the engineer suspects that many are irrelevant. The engineer wants to automatically perform feature selection during training. Which hyperparameter should the engineer adjust to achieve this?

Accepted Answer

Set the l1 regularization hyperparameter to a positive value.. L1 regularization in Linear Learner adds a penalty equal to the absolute value of weights, which promotes sparsity and effectively performs feature selection by zeroing out irrelevant coefficients. L2 regularization only shrinks weights, while use_bias and target_precision do not influence feature selection. Thus, l1 is the correct hyperparameter.

Answer

Set the use_bias hyperparameter to false.

Answer

Set the target_precision hyperparameter to a high value.

Answer

Set the l2 regularization hyperparameter to a positive value.

Question 9

A machine learning engineer is using Amazon SageMaker to train a deep learning model for image segmentation using the built-in Semantic Segmentation algorithm. The training dataset consists of 10,000 images with pixel-level annotations. The engineer notices that the model's validation loss is much higher than the training loss after 50 epochs, and the model performs poorly on a held-out test set. Which SageMaker training configuration change is MOST likely to improve the model's generalization?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Adjust the resource limits for the training jobs, such as increasing the instance count or using larger instances.. The training jobs are failing due to resource exhaustion, which means they lack sufficient compute, memory, or disk. Increasing the instance count or using larger instances provides the necessary resources for each job to complete. This allows the tuning job to evaluate more configurations and find a better model. Other options do not address the root cause of the failures.

Answer

Reduce the number of hyperparameters in the search space.

Answer

Increase the maximum number of training jobs in the tuning job configuration.

Answer

Change the objective metric to 'validation:accuracy'.

Question 10

A company is using Amazon SageMaker to train a deep learning model for natural language processing. The training dataset is stored in Amazon S3 in CSV format and is 500 GB in size. The data scientist wants to optimize the training job to reduce training time and cost. Which two actions should the data scientist take? (Choose two.)

Accepted Answer

Convert the dataset to the RecordIO format using the SageMaker built-in tool and use Pipe mode for training.. To reduce training time and cost for large datasets, using Pipe mode with RecordIO format streams data efficiently from S3, reducing I/O overhead and startup time. Managed Spot Training leverages cheaper Spot Instances, significantly cutting costs. Together, they optimize both time and expense. Increasing epochs or using larger instances may increase cost, and EFS is not as efficient for this purpose.

Answer

Increase the number of epochs to ensure the model converges.

Answer

Use a larger instance type with more GPUs to speed up training.

Answer

Store the dataset in Amazon EFS and mount it to the training container.

Question 11

A data scientist is using Amazon SageMaker to train a deep learning model for image classification with a large dataset. The training job is taking an excessively long time, and the data scientist wants to optimize the training process without compromising model accuracy. The dataset is stored in Amazon S3, and the training instances are ml.p3.8xlarge. Which two actions should the data scientist take to reduce training time? (Choose two.)

Accepted Answer

Use SageMaker Distributed Training with the Horovod framework to parallelize training across multiple instances.. To reduce training time without sacrificing accuracy, the data scientist should improve data ingestion and parallelize computation. Pipe mode streams data efficiently, avoiding the overhead of downloading large datasets. Distributed training with Horovod splits the workload across multiple instances, leveraging parallel compute to shorten training duration. Both actions directly target the bottlenecks in large-scale deep learning training on SageMaker.

Answer

Reduce the size of the training dataset by randomly sampling 50% of the images.

Answer

Decrease the learning rate to allow the model to converge more smoothly.

Answer

Switch to a smaller instance type, such as ml.m5.large, to reduce cost and increase availability.

Question 12

A data scientist is tuning a SageMaker XGBoost model and wants to search over learning rate, max depth, and subsample simultaneously. The search space is large and each training job takes about 40 minutes. The scientist needs to minimize total tuning cost while still finding a strong configuration. Which SageMaker automatic model tuning strategy should be used?

Accepted Answer

Bayesian optimization. When each training job is expensive and the search space spans several interacting hyperparameters, a strategy that learns from completed trials is most cost-effective. Bayesian optimization fits a surrogate model to observed objective values and proposes promising candidates, reducing the number of full training runs needed. Random and grid search ignore trial outcomes, and Hyperband's early stopping is designed for cheaper, epoch-wise evaluations than the 40-minute jobs described.

Answer

Grid search

Answer

Hyperband

Answer

Random search

Question 13

A data scientist is using Amazon SageMaker to train a model with the built-in K-Means algorithm. The data scientist wants to determine the optimal number of clusters for a dataset. Which approach should the data scientist use?

Accepted Answer

Use the elbow method by plotting the within-cluster sum of squares against the number of clusters.. The elbow method is a widely used heuristic for determining the optimal number of clusters in K-Means. It plots the within-cluster sum of squares against k and looks for an elbow, where adding more clusters yields diminishing returns. Other options are either incorrect (silhouette should be maximized, AIC not applicable) or arbitrary (feature count).

Answer

Use the silhouette score and select the k with the lowest score.

Answer

Use the Akaike information criterion (AIC) and select the k with the highest AIC value.

Answer

Set the number of clusters to the number of features in the dataset.

Question 14

A data scientist is training a gradient boosting model using Amazon SageMaker's built-in XGBoost algorithm on a dataset with 100 features. After training, the model's performance on the validation set is poor, and the training loss is much lower than the validation loss. The data scientist suspects overfitting. Which of the following hyperparameter adjustments is most likely to reduce overfitting?

Accepted Answer

Increase the 'lambda' (L2 regularization) hyperparameter.. Overfitting is characterized by a large gap between training and validation performance. Increasing the L2 regularization hyperparameter 'lambda' penalizes complex models, reducing the model's ability to fit noise. This leads to a simpler model that generalizes better. Other options either increase model complexity or do not directly address overfitting, and some could even worsen it.

Answer

Increase the 'subsample' hyperparameter to 1.0.

Answer

Decrease the 'eta' (learning rate) and increase the number of rounds.

Answer

Increase the 'max_depth' hyperparameter.

Question 15

A data scientist is using Amazon SageMaker to train a deep learning model for natural language processing. The training job runs on multiple GPU instances and uses the SageMaker distributed data parallel library. The data scientist observes that the training throughput decreases significantly as the number of GPUs increases. What is the most likely cause of this performance degradation?

Accepted Answer

The all-reduce communication overhead is increasing faster than the computational savings from adding GPUs.. The correct answer is that all-reduce communication overhead is increasing faster than computational savings. In distributed data parallel training, each GPU computes gradients on its local batch, and these gradients must be averaged across all GPUs using all-reduce. As the number of GPUs grows, the all-reduce operation involves more participants and more data exchange, which can saturate network bandwidth, especially across nodes. This communication cost can dominate, reducing overall throughput. Other factors like dataset size, batch size, or learning rate do not directly cause throughput to drop as GPUs increase.

Answer

The learning rate is not scaled appropriately for the larger effective batch size.

Answer

The training dataset is too small for the number of GPUs, causing each GPU to process very few batches.

Answer

The batch size per GPU is too large, causing memory swapping on each GPU.