MLS-C01 Modeling Practice Test 48 — 15 Questions

Question 1

A data scientist is using Amazon SageMaker to train a linear regression model on a dataset with a single feature. The model's training loss is high, and the validation loss is similarly high, indicating underfitting. The data scientist suspects that the relationship between the feature and the target is not linear. Which action should the data scientist take to improve the model's performance?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Add polynomial features (e.g., squared and cubic terms) to the input data and retrain the linear model.. Underfitting with a linear model on a single feature suggests the true relationship is nonlinear. Adding polynomial features increases the hypothesis space, enabling the linear model to fit curves. This directly addresses the high bias. Other options either target overfitting or do not increase model capacity. Thus, polynomial feature engineering is the most effective solution.

Answer

Apply L2 regularization (ridge) to the linear regression model to reduce overfitting.

Answer

Collect more training data to provide more examples for the model to learn from.

Answer

Increase the learning rate for the optimization algorithm to converge faster.

Question 2

A company trains an XGBoost model with the SageMaker built-in algorithm on a tabular dataset containing a high-cardinality categorical feature with 40,000 distinct values. Training accuracy is excellent but validation accuracy is poor, and the team suspects the categorical encoding. Which change should the team make to improve generalization?

Accepted Answer

Replace one-hot encoding with a target or frequency-based encoding computed on the training split only, and pass it as a numeric feature to XGBoost.. One-hot encoding a categorical with tens of thousands of levels produces an extremely wide, sparse matrix where rare levels are effectively memorized. Encoding the feature as a compact numeric signal, such as a target or frequency statistic derived only from the training split, reduces dimensionality and prevents leakage, which narrows the train-validation gap.

Answer

Increase the number of boosting rounds substantially and keep the same one-hot encoding of the categorical feature.

Answer

Keep one-hot encoding but reduce the learning rate and set the subsample parameter to 1.0 to use all rows per tree.

Answer

Switch to the SageMaker linear learner algorithm and feed the same one-hot encoded matrix, relying on L1 regularization to select useful columns.

Question 3

A machine learning engineer is using Amazon SageMaker to train a model with the built-in K-Means algorithm. The dataset has 1 million records with 10 numerical features. The engineer wants to determine the optimal number of clusters (k) for the data. Which approach should the engineer use to select k?

Accepted Answer

Use the elbow method by plotting the within-cluster sum of squares against k and look for a bend in the curve.. The elbow method is a standard heuristic for choosing k in K-Means. It plots the within-cluster sum of squares against k and identifies the point where the rate of decrease sharply changes, indicating diminishing returns. This helps balance model complexity and clustering quality. Other methods like silhouette analysis can also be used, but they require selecting the maximum score.

Answer

Train multiple models with different values of k and select the one with the lowest training loss.

Answer

Use the silhouette score and select the k that minimizes it.

Answer

Set k equal to the number of features in the dataset.

Question 4

A data scientist is training a linear regression model using Amazon SageMaker's built-in Linear Learner algorithm. The model's training loss is decreasing but the validation loss starts to increase after a certain number of epochs. The data scientist wants to prevent overfitting and improve generalization. What should the data scientist do?

Accepted Answer

Enable early stopping in the training job to halt training when validation loss stops improving.. Overfitting occurs when a model learns noise in the training data, causing validation loss to increase. Early stopping is an effective regularization technique that halts training at the point of best validation performance. In SageMaker, Linear Learner supports early stopping through hyperparameters like early_stopping_patience and early_stopping_tolerance, which monitor validation loss and stop when no improvement is seen.

Answer

Reduce the size of the training dataset to make the model generalize better.

Answer

Increase the learning rate to help the model converge faster and avoid overfitting.

Answer

Add more features to the dataset to provide additional information to the model.

Question 5

A data scientist is using Amazon SageMaker to train a linear regression model on a dataset with a single feature. The model's training loss is high, and the validation loss is similarly high, indicating underfitting. The data scientist suspects that the relationship between the feature and the target is not linear. Which action should the data scientist take to improve the model's performance?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Increase the scale_pos_weight hyperparameter to a value greater than 1.. The scale_pos_weight hyperparameter in XGBoost is designed to handle imbalanced classes by weighting the positive class more heavily. Setting it to a value greater than 1 increases the influence of positive examples during training, which improves recall for the minority class. Other options do not directly target the imbalance issue.

Answer

Decrease the learning rate and increase the number of boosting rounds.

Answer

Increase the max_depth hyperparameter to allow deeper trees.

Answer

Set the objective hyperparameter to 'binary:logistic' and increase the eval_metric to 'auc'.

Question 6

A data scientist is training a linear learner model on a dataset with 50 features. The model is overfitting, and the scientist wants to apply L1 regularization to encourage sparsity in the model weights. Which SageMaker built-in Linear Learner hyperparameter should be adjusted to control the strength of L1 regularization?

Accepted Answer

l1. The SageMaker built-in Linear Learner algorithm provides 'l1' and 'l2' hyperparameters for regularization. To encourage sparsity, the scientist should increase the 'l1' value, which adds an L1 penalty to the loss function and drives many weights to zero. L2 regularization ('l2') reduces overfitting but does not produce sparse models. 'wd' is not a valid hyperparameter, and 'learning_rate' affects optimization speed, not regularization.

Answer

learning_rate

Answer

l2

Answer

wd

Question 7

A team is training a deep learning model with the SageMaker built-in Object Detection algorithm on a custom dataset and observes that validation mean average precision plateaus early while training loss keeps decreasing. The team wants to reduce overfitting without changing the algorithm. (Choose two.)

Accepted Answer

Enable early stopping on the validation mean average precision metric so training halts when that metric stops improving.. Overfitting in the built-in Object Detection algorithm is countered by increasing effective data diversity and by stopping training at the right time. Augmentation hyperparameters such as random flip, crop, and color jitter broaden the input distribution, while early stopping on validation mean average precision halts training once the metric stops improving.

Answer

Increase the number of training epochs while keeping all other hyperparameters unchanged.

Answer

Reduce the mini-batch size to one so that gradient updates are computed per individual image.

Answer

Remove the validation dataset from training and evaluate only on the training set to confirm the model is learning.

Question 8

A machine learning engineer is using Amazon SageMaker to train a model with the built-in Object2Vec algorithm to learn embeddings for a recommendation system. The training data consists of user-item interaction pairs. The engineer wants to evaluate the quality of the learned embeddings. Which metric or method is most appropriate for assessing the embeddings' usefulness for downstream tasks?

Accepted Answer

Use the embeddings as features in a downstream classifier and evaluate its performance on a held-out test set.. The most reliable way to assess embedding quality is to evaluate their performance in a downstream task. For recommendation systems, this could involve using the embeddings to predict user-item interactions and measuring metrics like recall@k or NDCG. This approach directly tests whether the embeddings capture meaningful patterns that improve task outcomes, rather than relying on intrinsic metrics that may not correlate with practical utility.

Answer

Compute the average cosine similarity between all pairs of embeddings to ensure they are diverse.

Answer

Monitor the training loss curve and ensure it converges to a low value.

Answer

Visualize the embeddings using t-SNE and check for clear cluster separation.

Question 9

A machine learning specialist is using Amazon SageMaker to train a model on a dataset with a highly skewed target distribution. The specialist wants to evaluate the model's performance using metrics that are robust to class imbalance. Which metric should be used?

Accepted Answer

Area Under the Precision-Recall Curve (AUC-PR). AUC-PR is specifically designed to evaluate performance on the positive class in imbalanced datasets. It summarizes the trade-off between precision and recall, making it sensitive to improvements in minority class prediction. Other metrics like accuracy and AUC-ROC can be overly optimistic when the majority class dominates.

Answer

Mean Squared Error (MSE)

Answer

Accuracy

Answer

Area Under the Receiver Operating Characteristic Curve (AUC-ROC)

Question 10

A machine learning engineer is using SageMaker to train a built-in K-Means clustering model on a dataset of customer behavior features. The engineer wants to determine the optimal number of clusters (k) and plans to use the elbow method by training multiple models with different k values. Which SageMaker K-Means hyperparameter should be configured to ensure that the cluster quality metric is computed and available for analysis?

Accepted Answer

eval_metrics. The SageMaker built-in K-Means algorithm provides the 'eval_metrics' hyperparameter, which accepts a list of metrics to compute on the test dataset. To use the elbow method, the engineer must enable 'msd' (mean squared distance to centroid) so that the algorithm logs this value for each trained model. By comparing msd across different k values, the engineer can identify the elbow point. Other hyperparameters like k, mini_batch_size, and feature_dim do not control metric computation.

Answer

feature_dim

Answer

k

Answer

mini_batch_size

Question 11

A machine learning engineer is training a linear regression model on a dataset with 20 features. The training loss decreases but the validation loss increases after a few epochs. The engineer wants to apply a regularization technique that encourages sparsity in the feature coefficients, effectively performing feature selection. Which regularization method should be used?

Accepted Answer

L1 regularization (Lasso). The correct answer is L1 regularization (Lasso) because it induces sparsity by driving some coefficients to zero, effectively selecting a subset of features. This addresses the overfitting issue and meets the goal of feature selection. L2 regularization only shrinks coefficients, dropout is for neural networks, and early stopping does not affect model sparsity.

Answer

L2 regularization (Ridge)

Answer

Early stopping

Answer

Dropout

Question 12

A machine learning engineer is training a linear regression model in Amazon SageMaker using the built-in Linear Learner algorithm. The dataset contains a highly skewed target variable with extreme outliers. The engineer wants to reduce the influence of these outliers on the model without removing any data points. Which of the following approaches should the engineer take?

Accepted Answer

Apply a logarithmic transformation to the target variable before training.. The correct approach is to apply a logarithmic transformation to the target variable. Linear Learner minimizes squared error, which is highly sensitive to outliers. By compressing the range of the target, the transformation reduces the impact of extreme values on the loss, leading to a more robust model. Other options either do not address the data distribution or are inappropriate for regression.

Answer

Increase the number of epochs in the training job.

Answer

Set the 'predictor_type' hyperparameter to 'binary_classifier'.

Answer

Use a larger instance type for training.

Question 13

A financial services company is training a binary classification model on Amazon SageMaker to detect fraudulent transactions. The dataset is highly imbalanced: only 0.5% of transactions are fraudulent. The data scientist used the XGBoost built-in algorithm with default hyperparameters and achieved 99.5% accuracy on the validation set, but the model fails to identify any fraudulent cases in production. Which SageMaker training configuration change should the data scientist make to improve fraud detection?

Accepted Answer

Set the scale_pos_weight hyperparameter in the XGBoost algorithm to the ratio of negative to positive examples.. For highly imbalanced binary classification, adjusting the class weights is a standard technique. The XGBoost algorithm in SageMaker provides the scale_pos_weight hyperparameter, which sets the weight for the positive class. Setting it to the ratio of negative to positive examples (approximately 199) gives more importance to the minority class during training, enabling the model to detect fraud. Other options do not directly address the imbalance.

Answer

Apply L1 regularization to reduce overfitting and improve generalization.

Answer

Increase the number of training epochs to allow the model to learn the minority class better.

Answer

Use a larger instance type for training to speed up computation and allow more complex models.

Question 14

A data scientist is using Amazon SageMaker to train a linear regression model on a dataset with a single feature. The model's training loss is high, and the validation loss is similarly high, indicating underfitting. The data scientist suspects that the relationship between the feature and the target is not linear. Which action should the data scientist take to improve the model's performance?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Increase the dropout_ratio hyperparameter.. Increasing dropout_ratio and weight_decay are both regularization techniques that directly reduce overfitting. Dropout prevents complex co-adaptations, while weight decay penalizes large weights. Together, they improve generalization without requiring additional data. Other options do not specifically target overfitting.

Answer

Increase the number of epochs.

Answer

Increase the batch_size hyperparameter.

Answer

Decrease the learning_rate hyperparameter.

Question 15

A machine learning engineer is training a gradient boosting model on Amazon SageMaker for a binary classification task where the positive class is rare (1% of the data). The engineer wants the model to focus more on the minority class during training. Which SageMaker built-in algorithm hyperparameter should the engineer adjust to directly control the weight of the positive class?

Accepted Answer

Set the scale_pos_weight hyperparameter to a high value.. For imbalanced binary classification with SageMaker's built-in XGBoost, the scale_pos_weight hyperparameter is specifically designed to weight the positive class. Setting it to the ratio of negative to positive samples (e.g., 99 for 1% positives) helps the model pay more attention to the minority class. Other hyperparameters like max_depth, eta, and subsample do not directly control class weighting.

Answer

Set the max_depth hyperparameter to a high value.

Answer

Set the subsample hyperparameter to a low value.

Answer

Set the eta hyperparameter to a high value.