MLS-C01 Modeling Practice Test 52 — 15 Questions

Question 1

A data scientist is using Amazon SageMaker's built-in XGBoost algorithm to train a binary classification model. The dataset is highly imbalanced, with only 2% positive examples. After training with default parameters, the model predicts almost all examples as negative. Which modification to the training job is MOST likely to improve the model's ability to detect positive examples?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Set the scale_pos_weight hyperparameter to the ratio of negative to positive examples.. Setting scale_pos_weight to the ratio of negative to positive examples is the standard XGBoost method to handle class imbalance. It effectively increases the cost of misclassifying positive examples, pushing the model to improve recall for the minority class. Other hyperparameters may help optimization but do not directly address the imbalance.

Answer

Increase the max_depth hyperparameter to allow more complex trees.

Answer

Use the 'binary:logistic' objective with the 'auc' evaluation metric.

Answer

Decrease the learning_rate and increase the number of boosting rounds.

Question 2

A machine learning engineer is developing a model to predict whether a customer will churn within the next month. The dataset contains 10,000 records with 50 features, and the target variable is binary. The engineer wants to quickly establish a baseline and compare multiple algorithms with minimal coding effort. Which Amazon SageMaker built-in algorithm should the engineer use for this task?

Accepted Answer

Linear Learner. For a binary classification problem like churn prediction with tabular data, Linear Learner is a built-in SageMaker algorithm that provides a quick baseline. It supports supervised learning and can handle the dataset size and feature count without extensive preprocessing. Other options are unsupervised or dimensionality reduction techniques, which do not directly predict a binary target.

Answer

Random Cut Forest

Answer

Principal Component Analysis (PCA)

Answer

K-Means

Question 3

A machine learning engineer is preparing a dataset for training a model using Amazon SageMaker. The dataset contains a categorical feature with high cardinality (over 10,000 unique values) and a numerical feature with a long-tailed distribution. The engineer wants to apply appropriate feature engineering to improve model performance. (Choose two.)

Accepted Answer

Apply a logarithmic transformation to the numerical feature.. Target encoding reduces the dimensionality of a high-cardinality categorical feature while retaining its predictive signal, and a log transformation addresses the skewness of a long-tailed numerical feature. These two techniques are appropriate for the described dataset and can be applied in preprocessing steps before training with Amazon SageMaker.

Answer

Apply one-hot encoding to the high-cardinality categorical feature.

Answer

Apply min-max scaling to the numerical feature.

Answer

Apply principal component analysis (PCA) to the categorical feature.

Question 4

A machine learning engineer is training a gradient boosting model on Amazon SageMaker using the built-in XGBoost algorithm. The training dataset contains 2 million rows and 50 features. The engineer notices that the model's validation AUC is significantly lower than the training AUC, indicating overfitting. Which hyperparameter adjustment is MOST likely to reduce this overfitting?

Accepted Answer

Increase the value of alpha (L1 regularization).. The model shows a large gap between training and validation AUC, a classic overfitting signal. Increasing L1 regularization (alpha) penalizes complex leaf weights, shrinking the model and improving generalization. Other options either increase model complexity (max_depth, subsample=1.0) or accelerate fitting without regularization (eta), which would not address the core issue. Regularization strength is the direct lever for controlling overfitting in XGBoost.

Answer

Increase the value of eta (learning rate).

Answer

Increase the value of max_depth.

Answer

Increase the value of subsample to 1.0 and disable column sampling.

Question 5

A data scientist is using Amazon SageMaker's built-in XGBoost algorithm to train a binary classification model. The dataset is highly imbalanced, with only 2% positive examples. After training with default parameters, the model predicts almost all examples as negative. Which modification to the training job is MOST likely to improve the model's ability to detect positive examples?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Train multiple K-Means models with different values of k and use the elbow method by plotting the 'test:msd' against k, then select the k at the elbow point.. To determine the optimal number of clusters in K-Means, the elbow method is commonly used. It involves training models with varying k and plotting the mean squared distance to centroid (test:msd) against k. The point where the rate of decrease sharply changes—the elbow—suggests a good balance between cluster compactness and number of clusters. SageMaker's K-Means algorithm outputs this metric, making it straightforward to implement.

Answer

Train a single K-Means model with a large k (e.g., 100) and then merge clusters based on their centroid distances until the desired number is reached.

Answer

Train multiple K-Means models with different values of k and compare the 'test:msd' metric (mean squared distance to centroid) from the training logs, selecting the k with the lowest value.

Answer

Use the SageMaker Automatic Model Tuning feature to tune k, with the objective metric set to 'test:msd' and the goal to minimize it.

Question 6

A machine learning engineer is using Amazon SageMaker to train a deep learning model for natural language processing. The model is trained on a large corpus and achieves high accuracy on the training set but poor performance on the validation set. The engineer suspects overfitting and wants to apply regularization techniques. Which two of the following SageMaker features or techniques can help mitigate overfitting? (Choose two.)

Accepted Answer

Use dropout layers in the neural network architecture.. Early stopping and dropout are both effective regularization techniques for deep learning. Early stopping prevents the model from overfitting by halting training when validation performance degrades. Dropout introduces randomness during training, forcing the network to learn redundant representations and reducing overfitting. Together, they address the high training accuracy and low validation accuracy observed.

Answer

Increase the learning rate.

Answer

Use a larger batch size.

Answer

Increase the number of epochs for training.

Question 7

A data scientist is using Amazon SageMaker's built-in XGBoost algorithm to train a binary classification model. The dataset is highly imbalanced, with only 2% positive examples. After training with default parameters, the model predicts almost all examples as negative. Which modification to the training job is MOST likely to improve the model's ability to detect positive examples?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use SageMaker's Pipe mode with the dataset stored in Amazon S3 and set the input mode to 'Pipe'.. Pipe mode streams data directly from S3 to the training algorithm, avoiding the need to download the entire dataset to local storage before training. This reduces startup time and allows training to begin almost immediately. It also overlaps data loading with computation, potentially reducing overall training time. For a 500 GB dataset, this is the most efficient approach. The data scientist should verify that the chosen algorithm supports Pipe mode.

Answer

Use SageMaker's File mode with the dataset stored in Amazon EFS and mount it to the training instance.

Answer

Use SageMaker's FastFile mode with the dataset stored in Amazon S3 and set the input mode to 'FastFile'.

Answer

Increase the size of the training instance's storage volume to accommodate the entire dataset faster.

Question 8

A data scientist is tuning an Amazon SageMaker training job for a regression model. The team wants to automatically stop training runs that are no longer improving the validation metric and reduce total compute cost across many trials. They are using the SageMaker automatic model tuning feature with a Bayesian strategy. Which configuration should they apply?

Accepted Answer

Enable early stopping with a small max_no_improvement_epochs relative to the training duration. SageMaker automatic model tuning can stop individual trials early when the objective metric stops improving, which saves compute across a large search. Configuring early stopping with an appropriate max_no_improvement_epochs lets unproductive trials terminate quickly, while the Bayesian strategy continues to propose better configurations from completed trials.

Answer

Reduce the hyperparameter ranges and disable the tuner's warm start

Answer

Increase max_parallel_jobs to the total number of trials so all trials run at once

Answer

Set the objective metric to a training-set metric and rely on the tuner to stop bad trials

Question 9

A financial institution is building a fraud detection model using Amazon SageMaker. The dataset is highly imbalanced: only 0.5% of transactions are fraudulent. The team wants to evaluate model performance in a way that reflects the business cost of missing a fraudulent transaction, which is 100 times higher than the cost of a false alarm. Which evaluation strategy is MOST appropriate?

Accepted Answer

Use a cost-sensitive metric such as weighted false positive and false negative rates, or optimize expected monetary cost.. The key requirement is to reflect the 100:1 cost asymmetry between false negatives and false positives. A cost-sensitive evaluation metric, such as expected monetary cost or a weighted error rate, directly incorporates that asymmetry and supports threshold selection that minimizes business loss. Accuracy, ROC-AUC, and PR-AUC do not encode the specific cost ratio, so they cannot guarantee alignment with the stated business objective.

Answer

Use area under the ROC curve (AUC-ROC) and select the model with the highest AUC.

Answer

Use accuracy as the primary metric and optimize for it.

Answer

Use precision-recall AUC (PR-AUC) and select the model with the highest PR-AUC.

Question 10

A data scientist is using Amazon SageMaker's built-in XGBoost algorithm to train a binary classification model. The dataset is highly imbalanced, with only 2% positive examples. After training with default parameters, the model predicts almost all examples as negative. Which modification to the training job is MOST likely to improve the model's ability to detect positive examples?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use SageMaker automatic model tuning (hyperparameter tuning job) with the linear learner algorithm and specify a range for the L1 or L2 regularization parameter.. To automatically find the best hyperparameters while controlling overfitting, SageMaker automatic model tuning is the right service. By defining a hyperparameter range for the regularization parameter of the linear learner algorithm, the tuning job explores combinations and selects the one that optimizes the objective metric, effectively balancing fit and complexity. Other options either lack automatic search or do not directly address overfitting through regularization.

Answer

Use SageMaker Debugger to monitor training and automatically adjust hyperparameters.

Answer

Use the built-in linear learner with default hyperparameters and enable early stopping.

Answer

Use SageMaker Experiments to track different manual training runs and pick the best one.

Question 11

A machine learning engineer must improve a SageMaker-built XGBoost model that currently overfits a high-dimensional sparse text feature set with 200,000 features and 20,000 documents. The engineer wants to reduce dimensionality and improve generalization. (Choose two.)

Accepted Answer

Use colsample_bytree to subsample a fraction of features per tree. Sparse, very high-dimensional feature sets let tree models carve out tiny leaves keyed to rare tokens. Raising min_child_weight blocks splits that create underpopulated leaves, and colsample_bytree restricts each tree to a random feature subset, both reducing variance. These two SageMaker XGBoost hyperparameters directly target the overfitting and dimensionality concerns described.

Answer

Increase the number of boosting rounds without any other change

Answer

Set the eval_metric to mape and select the best iteration by that metric

Answer

Switch the objective to multi:softmax

Question 12

A data scientist is training a binary classification model using Amazon SageMaker's built-in Linear Learner algorithm. The dataset is highly imbalanced, with only 2% of the samples belonging to the positive class. The data scientist wants to adjust the algorithm to better handle the imbalance. Which hyperparameter should be modified?

Accepted Answer

Set the positive_example_weight_mult parameter to a value greater than 1.. The SageMaker Linear Learner algorithm provides the positive_example_weight_mult hyperparameter to handle imbalanced datasets by weighting the positive class more heavily. This is the correct approach because it directly influences the loss function to penalize misclassifications of the minority class more. The other options reference non-existent parameters or do not effectively address the imbalance.

Answer

Set the target_recall parameter to a high value.

Answer

Set the loss parameter to 'logistic' and increase the learning rate.

Answer

Set the balance_multiplier parameter to a value less than 1.

Question 13

A machine learning engineer is using Amazon SageMaker to train a model for predicting housing prices. The dataset contains 20,000 records with 15 numerical features. The engineer notices that the model's predictions are consistently off by a large margin for high-priced houses. Which SageMaker built-in algorithm is BEST suited for this regression task?

Accepted Answer

XGBoost. For regression on tabular data with potential nonlinear relationships, XGBoost is a strong choice. It can model complex interactions and is robust to outliers. Linear Regression may underfit, while K-Means and DeepAR are for unsupervised and time-series tasks, respectively.

Answer

DeepAR

Answer

K-Means

Answer

Linear Regression

Question 14

A data scientist is using the Amazon SageMaker k-means built-in algorithm to segment customers based on 12 numerical behavioral features. Before training, the scientist must decide how to prepare the data and how to choose the number of clusters. Which approach is most appropriate?

Accepted Answer

Standardize the features and choose k by comparing within-cluster sum of squares across candidate values. k-means relies on Euclidean distance, so standardizing features ensures equal influence across attributes measured in different units. Selecting k by examining within-cluster sum of squares for several candidates identifies the elbow where additional clusters stop adding meaningful separation. This combination is the conventional, effective preparation for the SageMaker k-means algorithm.

Answer

Leave features unscaled and set k equal to the number of features

Answer

Apply one-hot encoding to all numeric features and use the default k

Answer

Use principal component analysis to reduce to two dimensions, then set k to two

Question 15

A machine learning team is using Amazon SageMaker to train a deep learning model for image segmentation. They notice that the model's performance on the validation set is much worse than on the training set, even after adding dropout and data augmentation. They want to further reduce overfitting without collecting more data. Which technique is MOST likely to help?

Accepted Answer

Apply weight decay (L2 regularization) to the convolutional and fully connected layers.. When dropout and data augmentation have not sufficiently reduced overfitting, adding weight decay (L2 regularization) is a direct way to penalize complex weights and improve generalization. It constrains the model's effective capacity without new data. Increasing epochs or merely lowering the learning rate does not address the underlying complexity, and batch normalization is not primarily an overfitting remedy. Weight decay is a well-established, effective next step.

Answer

Increase the number of epochs.

Answer

Add batch normalization layers.

Answer

Reduce the learning rate and train for more epochs.