MLS-C01 Modeling Practice Test 31 — 15 Questions

Question 1

A data scientist is training a regression model. The training loss is decreasing but the validation loss starts to increase after a few epochs. Which technique should the scientist use to address this issue?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Implement early stopping based on validation loss.. The described pattern — training loss decreasing while validation loss increases — is the textbook signature of overfitting. Early stopping monitors validation loss and halts training when it stops improving (often with a patience parameter), preventing the model from memorizing the training data. This directly addresses the divergence between training and validation performance.

Answer

Decrease the batch size.

Answer

Add more layers to the model.

Answer

Increase the learning rate.

Question 2

Which TWO of the following are best practices for hyperparameter tuning using Amazon SageMaker? (Choose 2)

Accepted Answer

Use early stopping to terminate poorly performing training jobs.. Option B is correct because SageMaker automatic model tuning supports early stopping, which stops underperforming training jobs before they run all epochs, saving time and compute cost while still allowing the tuner to focus resources on promising hyperparameter combinations. Option E is correct because SageMaker's automatic model tuning (the CreateHyperParameterTuningJob API) uses Bayesian optimization by default, which intelligently searches the hyperparameter space and typically finds good configurations with fewer training jobs than exhaustive methods. Option A is not a best practice because grid search exhaustively evaluates every combination, which is computationally expensive and inefficient compared with SageMaker's Bayesian or random search strategies. Option C is incorrect because including every algorithm hyperparameter bloats the search space and slows tuning; you should tune only the hyperparameters that materially affect model performance. Option D is incorrect because increasing the training dataset size is a data strategy, not a hyperparameter tuning best practice, and it does not by itself improve the tuning process.

Answer

Use grid search to exhaustively explore all combinations.

Answer

Include all algorithm hyperparameters in the tuning job.

Answer

Use a larger training dataset to improve tuning results.

Question 3

Which THREE of the following are valid strategies to reduce overfitting in a deep neural network? (Choose 3)

Accepted Answer

Use early stopping.. Option B (Use early stopping) is correct because monitoring validation loss and halting training when it stops improving prevents the model from continuing to fit noise in the training set, which is a classic overfitting-reduction technique. Option D (Add L2 regularization to the loss function) is correct because L2 adds a penalty proportional to the sum of squared weights, shrinking weights toward zero and constraining model complexity, thereby reducing overfitting. Option E (Use dropout layers) is correct because dropout randomly deactivates a fraction of neurons during each training step, forcing the network to learn redundant, robust features and acting as an ensemble-like regularizer. Option A (Increase the number of layers) is not correct because adding layers increases model capacity, which typically worsens overfitting rather than reducing it. Option C (Increase the learning rate) is not correct because a larger learning rate affects optimization speed and stability, not regularization, and can even lead to divergence or poorer generalization rather than reducing overfitting.

Answer

Increase the number of layers.

Answer

Increase the learning rate.

Question 4

Which TWO of the following are appropriate use cases for using Amazon SageMaker BlazingText? (Choose 2)

Accepted Answer

Text classification using supervised learning.. Option A (text classification using supervised learning) is correct because BlazingText implements a supervised mode that trains a classifier on labeled text, supporting both text classification and multi-label classification with fast GPU-accelerated training. Option C (learning word embeddings from a large text corpus) is correct because BlazingText's unsupervised mode implements Word2Vec (skip-gram and CBOW with negative sampling) to produce dense word vectors from unlabeled text. Option B is not appropriate because time series forecasting is handled by Amazon Forecast or SageMaker's DeepAR, not BlazingText. Option D is not appropriate because image classification requires a vision model such as SageMaker's Image Classification algorithm or a CNN, whereas BlazingText operates on text tokens. Option E is not appropriate because sequence-to-sequence translation is performed by neural machine translation models (e.g., SageMaker Seq2Seq or Transformer), not by BlazingText's classification/embedding algorithms.

Answer

Time series forecasting.

Answer

Classifying images.

Answer

Sequence-to-sequence translation.

Question 5

A retail company uses Amazon SageMaker to train a model for product demand forecasting. The dataset contains daily sales data for 10,000 products over 3 years. The data includes features like price, promotions, holidays, and seasonality. The data scientist uses a linear regression model and gets an RMSE of 50 units. However, the business requires more accurate forecasts, especially for products with high variability. The scientist notices that the residuals show a pattern: the model underestimates demand during promotional periods. Which approach should the scientist take to improve the model?

Accepted Answer

Add interaction features between promotion and other variables.. The residuals show a systematic pattern where the model underestimates demand during promotions, indicating the linear model cannot capture the interaction between promotions and other features. Adding interaction features (e.g., promotion × price, promotion × holiday) allows the linear model to represent these conditional relationships, improving accuracy for high-variability products. This is a targeted feature engineering fix that directly addresses the observed bias.

Answer

Collect more historical data for training.

Answer

Use a deep learning model like LSTM.

Answer

Remove promotion features to simplify the model.

Question 6

A machine learning engineer is using SageMaker to train an XGBoost model on a dataset with a severe class imbalance (1:1000). The goal is to maximize recall on the minority class. Which hyperparameter tuning strategy is MOST appropriate?

Accepted Answer

Set scale_pos_weight to the ratio of negative to positive samples. scale_pos_weight is XGBoost's dedicated hyperparameter for class imbalance; setting it to the ratio of negative to positive samples (roughly 1000 in this case) reweights the positive class's gradient contribution so the model prioritizes minority-class recall. This directly addresses the 1:1000 imbalance at the loss-function level, which is the most targeted strategy for maximizing recall on the minority class.

Answer

Set max_delta_step to a high value

Answer

Increase subsample ratio to 1.0

Answer

Set objective to 'binary:logistic' and tune max_depth

Question 7

A data scientist is evaluating a regression model. The RMSE on the training set is 2.5, and on the test set is 2.7. The R² on the test set is 0.98. What does this indicate?

Accepted Answer

The model generalizes well with no severe overfitting. The training RMSE (2.5) and test RMSE (2.7) are very close, and the test R² of 0.98 is high, indicating the model explains 98% of the variance on unseen data. A small gap between train and test error is the hallmark of good generalization. There is no evidence of severe overfitting or high bias.

Answer

The model has high bias

Answer

The model is underfitting because R² is too high

Answer

The model is overfitting because RMSE is lower on training data

Question 8

A company uses SageMaker to host a real-time inference endpoint for a classification model. The endpoint receives traffic spikes that cause high latency. The team wants a solution that automatically scales based on demand while keeping costs low. Which approach is BEST?

Accepted Answer

Enable automatic scaling on the endpoint using Application Auto Scaling. Application Auto Scaling for SageMaker endpoints automatically adjusts the number of instances provisioned for a model in response to changes in workload, such as traffic spikes. This enables the endpoint to scale out during high demand and scale in during low demand, maintaining low latency while minimizing costs. It is the native AWS solution for dynamic scaling of SageMaker real-time inference endpoints.

Answer

Use provisioned concurrency for the endpoint

Answer

Use a multi-model endpoint to serve multiple models

Answer

Deploy the endpoint on Spot Instances

Question 9

A data scientist is building a binary classifier and obtains the following confusion matrix on the test set: TP=80, FP=20, TN=70, FN=30. What is the precision?

Accepted Answer

0.8. Precision is defined as TP / (TP + FP), which measures the proportion of positive predictions that are actually correct. Plugging in the values: 80 / (80 + 20) = 80 / 100 = 0.8. This matches option B exactly, confirming it as the correct answer.

Answer

0.727

Answer

0.75

Answer

0.762

Question 10

A team is training a large language model using SageMaker's distributed training. They notice that the training loss is not decreasing after the first few epochs. Which action is MOST likely to resolve this issue?

Accepted Answer

Reduce the learning rate. When training loss plateaus or does not decrease after the first few epochs, a common cause is that the learning rate is too high, causing the optimizer to overshoot minima. Reducing the learning rate helps the model converge. Increasing the batch size (option A) mainly affects gradient variance and training speed but does not address an overly large step size. Adding L2 regularization (option B) helps prevent overfitting but does not resolve a high learning rate. Switching from Adam to SGD (option D) may not help because Adam typically adapts learning rates per parameter; if the base learning rate is too high, both optimizers can struggle. Therefore, reducing the learning rate is the most direct and effective action.

Answer

Increase the batch size

Answer

Add L2 regularization

Answer

Switch from Adam to SGD optimizer

Question 11

A company is deploying a model that predicts customer churn. The model's recall for the churn class is 0.9, but precision is 0.4. The business cost of false positives is high. Which strategy would MOST likely improve precision without significantly harming recall?

Accepted Answer

Increase the decision threshold for the churn class. Adjusting the decision threshold to require a higher probability before predicting churn can reduce false positives (increase precision) but may lower recall. The goal is to find a threshold that balances both. Using more aggressive regularization or different algorithms may not directly control the trade-off.

Answer

Collect more data for the churn class

Answer

Use a different algorithm such as Random Forest

Answer

Decrease the decision threshold for the churn class

Question 12

A data scientist is training a linear regression model and wants to handle multicollinearity among features. Which TWO actions are appropriate?

Accepted Answer

Use Ridge regression (L2 regularization). Ridge regression (L2 regularization) is correct because its penalty term shrinks coefficients toward zero, which stabilizes estimates when predictors are highly correlated and reduces the variance inflation caused by multicollinearity. Removing one of the highly correlated features is also correct because eliminating a redundant predictor breaks the near-linear dependency among inputs, directly addressing the source of multicollinearity. Adding interaction terms is not appropriate here since it introduces additional correlated predictors and typically worsens multicollinearity. Lasso (L1) can perform feature selection, but it is not the standard remedy for multicollinearity and can arbitrarily drop one of a correlated pair while producing unstable coefficient estimates. Scaling features to zero mean and unit variance changes units only and does not reduce correlation among predictors, so it does not resolve multicollinearity.

Answer

Add interaction terms between features

Answer

Use Lasso regression (L1 regularization)

Answer

Scale all features to have zero mean and unit variance

Question 13

A data scientist is evaluating a binary classification model. The model's AUC-ROC is 0.95. Which TWO statements are true?

Accepted Answer

The model has excellent discriminative ability. Option B is correct because an AUC-ROC of 0.95 indicates excellent discriminative ability: the model has a high probability of ranking a randomly chosen positive instance above a randomly chosen negative instance, with 0.5 being random and 1.0 being perfect. Option C is correct because AUC-ROC is computed by integrating the ROC curve across all possible classification thresholds, so it summarizes performance independent of any single decision threshold. Option A is wrong because AUC-ROC does not guarantee zero false positives; a model can have a high AUC while still producing false positives at certain thresholds. Option D is wrong because AUC-ROC measures ranking discrimination, not calibration, which concerns whether predicted probabilities match observed frequencies. Option E is wrong because a high AUC does not imply accuracy of at least 95%, especially under class imbalance or at a poorly chosen threshold.

Answer

The model has no false positives

Answer

The model is well-calibrated

Answer

The model's accuracy is at least 95%

Question 14

A data scientist is working on a regression problem to predict house prices. The dataset has 80 features, including categorical variables with high cardinality (e.g., zip code with 10,000 unique values). The target variable is log-transformed. The data scientist trains a linear regression model and obtains an R² of 0.45 on the test set. To improve performance, the data scientist considers: A) Applying one-hot encoding to all categorical features and using Ridge regression. B) Using target encoding for high-cardinality features and using a tree-based model like XGBoost. C) Removing all categorical features and using polynomial features for numerical features. D) Using principal component analysis (PCA) on all features before training a linear model. Which approach is MOST likely to improve the model's performance?

Accepted Answer

Target encoding + XGBoost. High-cardinality categorical features like zip code with 10,000 unique values make one-hot encoding explode dimensionality and sparsity, which hurts linear models. Target encoding replaces each category with a statistic (e.g., mean target) computed from training data, preserving signal in a single numeric column. XGBoost handles non-linear relationships and interactions well, and combined with target encoding it can capture the zip-code effect without the curse of dimensionality. This combination is the standard best practice for high-cardinality categorical regression.

Answer

Remove categorical features and use polynomial features

Answer

One-hot encoding + Ridge regression

Answer

PCA on all features before linear regression

Question 15

A data scientist is using Amazon SageMaker to train a model with the built-in XGBoost algorithm. The dataset contains missing values. What is the default behavior of SageMaker XGBoost regarding missing values?

Accepted Answer

It automatically learns the best direction (left or right) for missing values during training. SageMaker's built-in XGBoost algorithm handles missing values natively by learning a default direction for each tree split: during training, when a feature value is missing, the algorithm assigns it to the left or right branch based on which direction minimizes loss. This is the same sparse-aware behavior as open-source XGBoost, and it means no explicit imputation is required. The 'missing' parameter in SageMaker XGBoost defaults to 'NaN', enabling this automatic handling.

Answer

It raises an error and stops training

Answer

It imputes missing values with the column mean

Answer

It removes rows with missing values