MLS-C01 Modeling Practice Test 22 — 15 Questions

Question 1

A team is building a regression model to predict house prices. They observe that the model performs well on training data but poorly on validation data. Which THREE actions can help reduce overfitting? (Choose THREE.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Reduce model complexity by selecting fewer features. (reduce model complexity by selecting fewer features) reduces overfitting by limiting the model's capacity to learn noise. Option B (increase regularization strength) penalizes large coefficients, discouraging complex fits. Option C (collect more training data) provides more examples, helping the model generalize. Option D (increase maximum depth) increases model complexity, worsening overfitting. Option E (adding interaction features) increases complexity, likely increasing overfitting.

Answer

Increase the maximum depth of decision trees

Answer

Add more interaction features

Question 2

A machine learning engineer is tuning a Gradient Boosting model for a regression task. The dataset contains 50 features and 100,000 samples. The engineer wants to speed up training without sacrificing predictive performance significantly. Which THREE hyperparameters should the engineer consider adjusting? (Choose THREE.)

Accepted Answer

Reduce the subsample ratio (e.g., from 1.0 to 0.5). (subsample) uses a fraction of samples per tree, reducing both overfitting and training time. Option D (max_depth) limits tree depth, which directly reduces computation. Option E (max_features) restricts the number of features considered for each split, lowering tree complexity. Option B (learning_rate and n_estimators) trades off; although increasing learning rate can reduce training time, it often requires careful adjustment and may hurt performance. Option C (n_estimators) directly increases training time, which is the opposite of the goal.

Answer

Increase learning_rate and decrease n_estimators proportionally

Answer

Increase the number of estimators

Question 3

A data scientist is training a linear regression model using Amazon SageMaker's built-in Linear Learner algorithm. The dataset has 500 features and 1 million rows. After training, the model's training RMSE is 2.5 and validation RMSE is 2.6, which is acceptable. However, the scientist notices that many feature coefficients are very small but non-zero, and the model takes a long time to train. The scientist wants to reduce training time while maintaining similar accuracy. Which action should the scientist take?

Accepted Answer

Increase the L1 regularization strength. (increase L1 regularization) will drive many coefficients to zero, reducing effective features and thus training time. Option A (increase mini-batch size) may speed training but could affect convergence. Option C (switch to a neural network model) is unnecessary for this task. Option D (increase L2 regularization) shrinks coefficients but doesn't zero them out, so less impact on training speed.

Answer

Increase the mini-batch size

Answer

Switch to a neural network model

Answer

Increase the L2 regularization strength

Question 4

A company is building a fraud detection model using a random forest classifier. The dataset is highly imbalanced with 99% legitimate transactions and 1% fraudulent. The model currently achieves 99% accuracy on the test set, but the fraud recall is only 10%. The business requires at least 80% recall for fraud. The data scientist has tried oversampling the minority class and adjusting class weights, but recall remains below 40%. The dataset contains millions of transactions with hundreds of features. Which approach should the data scientist try next to improve fraud recall?

Accepted Answer

Use a gradient boosting machine (e.g., XGBoost) with scale_pos_weight parameter. The current model uses random forest but still has low recall after oversampling and class weights. Gradient boosting machines like XGBoost can handle imbalanced data well with the scale_pos_weight parameter, which adjusts the weight of the positive class. This approach often yields better recall than random forest on very imbalanced datasets. Option A (random undersampling) would discard too much majority data and reduce model performance. Option C (PCA) is not directly targeting the imbalance issue and may remove important features. Option D (logistic regression) is a linear model that typically underperforms on complex, high-dimensional data with severe imbalance.

Answer

Randomly undersample the majority class to a 50:50 ratio

Answer

Apply PCA to reduce dimensionality before training

Answer

Use a logistic regression model with L2 regularization

Question 5

A research team is training a deep learning model for image classification using Amazon SageMaker. The model is a convolutional neural network (CNN) with 50 layers. The team uses a single ml.p3.2xlarge instance. After 10 hours of training, the model has not converged and the loss is decreasing very slowly. The team suspects vanishing gradients. They want to diagnose and fix the issue without significant code changes. Which action should they take?

Accepted Answer

Modify the architecture to include residual connections (skip connections). (Modify the architecture to include residual connections) directly addresses vanishing gradients by allowing gradients to flow through skip connections. Option A (adding more layers) worsens the problem. Option C (batch normalization) helps but is not as targeted as residual connections. Option D (increase learning rate) may cause divergence.

Answer

Add more convolutional layers to increase model capacity

Answer

Use batch normalization after each convolutional layer

Answer

Increase the learning rate by a factor of 10

Question 6

A data scientist is building a binary classification model to predict whether a customer will subscribe to a service. The dataset contains 20 features, including categorical variables with high cardinality (e.g., zip code with 10,000 unique values). The scientist uses a logistic regression model and obtains a training AUC of 0.85 and a test AUC of 0.60. The scientist suspects overfitting due to high cardinality features. Which approach should the scientist use to address this issue?

Accepted Answer

Apply target encoding with smoothing to the zip code feature. (target encoding with smoothing) reduces cardinality while preserving predictive power. Option A (label encoding) may introduce ordinality issues. Option B (remove zip code) may lose important information. Option D (one-hot encoding) increases dimensionality drastically.

Answer

Apply label encoding to the zip code feature

Answer

Remove the zip code feature entirely

Answer

Apply one-hot encoding to the zip code feature

Question 7

A machine learning engineer is deploying a sentiment analysis model using Amazon SageMaker. The model is a BERT-based transformer that takes up to 512 tokens. The engineer notices that inference latency is high (over 500 ms per request) on a single ml.c5.xlarge instance. The application requires latency under 100 ms. The model has already been optimized using half-precision (FP16). Which action should the engineer take to reduce latency?

Accepted Answer

Use a GPU instance such as ml.g4dn.xlarge. (Use a GPU instance such as ml.g4dn.xlarge) provides the parallel processing power needed to accelerate transformer inference, significantly reducing latency for BERT-based models even after FP16 optimization. Option B (Reduce max sequence length to 128) would lower computational cost and latency but at the risk of truncating input and degrading accuracy, so it is not the primary recommended action. Option C (Increase batch size) improves throughput for multiple requests but does not reduce the latency of a single request. Option D (SageMaker Neo) optimizes the model for the target instance, but with an already optimized model (FP16) and strict latency requirement under 100 ms, switching to a GPU instance is more effective.

Answer

Reduce the maximum sequence length to 128

Answer

Increase the batch size for inference requests

Answer

Use SageMaker Neo to compile the model for the target instance

Question 8

A data scientist is training a time series forecasting model using Amazon SageMaker's DeepAR algorithm. The dataset contains daily sales data for 10,000 products over 2 years. The scientist splits the data chronologically: training on the first 18 months, validation on the next 3 months, and test on the last 3 months. The model performs well on validation but poorly on test. The data scientist suspects the model is overfitting to the validation period. Which action should the scientist take to improve test performance?

Accepted Answer

Use time series cross-validation with an expanding window. Time series cross-validation with an expanding window evaluates the model on multiple validation periods, ensuring robustness and reducing overfitting to a single validation window. Option B reduces context length and may lose long-term dependencies. Option C may introduce irrelevant features and does not address overfitting to validation. Option D removes validation entirely, which would not help diagnose or reduce overfitting.

Answer

Reduce the context length to 30 days

Answer

Add more exogenous features like holidays and promotions

Answer

Use the entire dataset for training and ignore validation

Question 9

A company is using Amazon SageMaker to train a linear learner model for predicting customer lifetime value. The target variable is right-skewed with a long tail. The data scientist applies a log transformation to the target variable and trains the model. The model achieves a low root mean squared error (RMSE) on the log scale. However, when the predictions are exponentiated back to the original scale, the RMSE is much higher. Which step should the data scientist take to improve the model's performance on the original scale?

Accepted Answer

Use a loss function that models the original distribution, such as Poisson or Tweedie. Using a loss function like Poisson or Tweedie directly models the non-negative, skewed distribution of the target variable in its original scale, which avoids the bias introduced by log transformation when predicting on the original scale. Option A (increase regularization) may not address the scale mismatch. Option B (remove outliers) could discard valuable data. Option D (use a deep learning model) might not solve the fundamental issue of loss function selection.

Answer

Increase the regularization strength

Answer

Remove outliers from the training data

Answer

Use a deep learning model instead of linear learner

Question 10

A data scientist is training a multiclass classification model to categorize support tickets into 50 categories. The dataset has 100,000 labeled tickets. The scientist uses a random forest classifier with 100 trees. The model achieves 90% accuracy on the test set, but the F1-score for some rare categories is below 0.1. The scientist wants to improve performance on rare categories without significantly reducing overall accuracy. Which approach should the scientist try?

Accepted Answer

Use class_weight='balanced' or compute custom class weights. (use class_weight='balanced' or compute custom class weights) helps the model focus on rare classes by assigning higher penalties to misclassifications of minority classes. Option B (reduce the number of trees to 50) may reduce model capacity and hurt overall performance. Option C (switch to a one-vs-rest logistic regression model) may not handle rare classes well without class weighting. Option A (increase the maximum depth of trees) could lead to overfitting and may not address class imbalance.

Answer

Increase the maximum depth of trees

Answer

Reduce the number of trees to 50 to prevent overfitting

Answer

Switch to a one-vs-rest logistic regression model

Question 11

A research team is training a deep learning model for object detection using SageMaker's built-in SSD algorithm. The dataset contains 50,000 images with bounding box annotations. The team uses a single ml.p3.2xlarge instance. After 24 hours of training, the model's loss has plateaued, but the mean average precision (mAP) on validation is only 0.45. The team wants to improve mAP without increasing training time. Which action should they take?

Accepted Answer

Use a pre-trained model as the backbone (e.g., ResNet-50 pre-trained on ImageNet). (use a pre-trained backbone) transfers learned features, often improving accuracy. Option A (increase learning rate) may destabilize training. Option C (increase batch size) may not improve mAP and could slow convergence. Option D (add more layers) increases training time.

Answer

Increase the learning rate by a factor of 2

Answer

Increase the batch size to 64

Answer

Add more convolutional layers to the backbone

Question 12

A data scientist is building a regression model to predict energy consumption. The dataset includes features like temperature, humidity, day of week, and holiday flags. The scientist uses a linear regression model and obtains an R-squared of 0.85 on training and 0.40 on test. The scientist suspects the model is not capturing non-linear relationships. Which approach should the scientist use to capture non-linearity?

Accepted Answer

Add polynomial features (e.g., squared terms and interactions). (add polynomial features) captures non-linear relationships by introducing squared terms and interactions, allowing the linear model to fit curvature. Option A (PCA) reduces dimensionality but does not add non-linearity. Option B (L1 regularization using Lasso) reduces overfitting by shrinking coefficients but does not introduce non-linear terms. Option C (removing features with low correlation) may lose information and does not help with non-linearity.

Answer

Apply PCA to the feature set

Answer

Increase L1 regularization using Lasso

Answer

Remove features with low correlation to the target

Question 13

A company is building a recommendation system using matrix factorization. The dataset has 1 million users and 100,000 items. The data scientist trains a model using SageMaker's Factorization Machines algorithm. The model achieves a root mean squared error (RMSE) of 0.95 on the test set. However, the business requires RMSE below 0.90. The data scientist has already tuned hyperparameters like number of factors and learning rate. Which additional step should the data scientist take to improve RMSE?

Accepted Answer

Add side features such as user demographics and item categories. (add side features) provides more information to the model, which can help reduce RMSE. Option B (more iterations) may not help if the model has already converged. Option C (increasing factors to 1000) could lead to overfitting. Option D (linear regression) is less powerful than matrix factorization for recommendation tasks.

Answer

Increase the number of training iterations

Answer

Increase the number of factors to 1000

Answer

Use a linear regression model instead

Question 14

A data scientist is training a binary classifier to detect network intrusions. The dataset has 1,000 features and 10 million samples, but only 0.1% are positive (intrusions). The scientist uses XGBoost with scale_pos_weight set to 100. The model achieves a recall of 0.90 and precision of 0.05 on the test set. The business requires precision of at least 0.50 while maintaining recall above 0.80. Which technique should the scientist apply?

Accepted Answer

Tune the decision threshold on validation data to maximize F1 score. Tuning the decision threshold on validation data adjusts the trade-off between precision and recall. The current model has high recall (0.90) but low precision (0.05). By raising the decision threshold, precision increases while recall decreases. Tuning on validation data allows finding a threshold that meets the business requirements of precision ≥0.50 and recall ≥0.80. Option A (switching to random forest) may not achieve the required precision. Option B (undersampling majority class) may reduce recall. Option D (increasing scale_pos_weight to 500) further increases recall and decreases precision.

Answer

Switch to a random forest classifier with class weights

Answer

Randomly undersample the majority class to achieve 1:1 ratio

Answer

Increase scale_pos_weight to 500

Question 15

A data scientist is training a binary classifier using a large dataset with class imbalance (90% negative, 10% positive). After training a logistic regression model, the F1 score is low but accuracy is high. Which TWO actions should the data scientist take to improve model performance? (Choose 2.)

Accepted Answer

Switch to evaluation metrics such as F1 score or AUC-ROC instead of accuracy.. (switch to evaluation metrics like F1 or AUC-ROC) is correct because accuracy is misleading for imbalanced datasets. Option D (resample using SMOTE) is correct because it balances the classes and helps the model learn the minority class better. Option B (feature scaling) is a general preprocessing step but does not directly address class imbalance. Option C (add more features) may not help if the model is already biased toward the majority class. Option E (increase regularization) might reduce overfitting but does not target the imbalance issue.

Answer

Apply feature scaling to ensure all features contribute equally.

Answer

Add more features to the model to improve its capacity.

Answer

Increase the regularization parameter to reduce overfitting.