MLS-C01 Modeling Practice Test 21 — 15 Questions

Question 1

A company uses an XGBoost model to predict equipment failures. The model has high precision but low recall. The business impact of a false negative is very high (missing a failure). Which action would MOST effectively increase recall while keeping precision reasonably high?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Decrease the probability threshold for the positive class. Decreasing the probability threshold for the positive class means the model will classify a case as a failure at a lower predicted probability, which captures more true positives (increases recall). However, this also allows more false positives, so precision may drop, but the trade-off is acceptable given the high cost of false negatives. This is a standard post-training calibration technique for imbalanced classification problems.

Answer

Increase the regularization parameter lambda

Answer

Set the objective to 'reg:squarederror'

Answer

Increase the number of boosting rounds

Question 2

Which THREE techniques can help reduce overfitting in a neural network trained on a small dataset?

Accepted Answer

Apply L2 weight regularization. L2 weight regularization (also known as weight decay) penalizes large weights by adding a term to the loss function proportional to the sum of squared weights. This forces the network to learn simpler patterns and reduces sensitivity to noise in the training data, which is especially helpful when the dataset is small and prone to overfitting.

Answer

Increase the number of hidden layers

Answer

Train for more epochs

Question 3

A machine learning team is training a deep learning model on Amazon SageMaker and notices that the training loss is decreasing but the validation loss is increasing. What is the most likely cause?

Accepted Answer

Overfitting the training data. When training loss continues to decrease while validation loss increases, the model is memorizing the training data rather than learning generalizable patterns. This is the classic symptom of overfitting, where the model's capacity exceeds what is needed for the underlying data distribution, causing it to fit noise in the training set. In Amazon SageMaker, this can be observed by monitoring the validation loss metric during training jobs.

Answer

Vanishing gradients

Answer

Learning rate is too high

Answer

Underfitting the training data

Question 4

A data scientist is tuning a gradient boosting model using Amazon SageMaker's Automatic Model Tuning (hyperparameter optimization). The objective metric is validation:auc. After 50 training jobs, the best model still has a validation AUC of only 0.65. The scientist suspects overfitting because the training AUC is 0.99. Which hyperparameter configuration is MOST likely to reduce overfitting?

Accepted Answer

Increase lambda from 1 to 10. Increasing lambda (L2 regularization) from 1 to 10 adds a stronger penalty on the magnitude of leaf weights in the gradient boosting model. This directly reduces overfitting by discouraging the model from fitting noise in the training data, which is consistent with the observed gap between training AUC (0.99) and validation AUC (0.65). In XGBoost, lambda controls the L2 regularization term on weights, and a higher value forces the model to be simpler and more generalizable.

Answer

Increase num_round from 100 to 500

Answer

Increase max_depth from 6 to 12

Answer

Increase subsample from 0.5 to 1.0

Question 5

A data scientist is training a neural network for image classification. The training loss is not decreasing significantly, and the validation loss is high. Which TWO actions should the scientist take to address potential vanishing gradients?

Accepted Answer

Use ReLU activation functions in hidden layers. ReLU activation functions help mitigate vanishing gradients because they output a constant gradient of 1 for positive inputs, preventing the gradient from shrinking as it propagates backward through many layers. This avoids the exponential decay of gradients that occurs with saturating activations like sigmoid or tanh, enabling effective training of deep networks.

Answer

Increase the learning rate

Answer

Switch activation functions from ReLU to sigmoid

Answer

Remove dropout layers

Question 6

A company is using Amazon SageMaker to train a large language model. The training job is taking too long. The data scientist wants to reduce training time without sacrificing model accuracy. Which THREE strategies are MOST appropriate?

Accepted Answer

Use mixed precision training (float16). Mixed precision training (float16) reduces memory usage and accelerates computation by using half-precision floating-point numbers for most operations, while maintaining a single-precision copy of critical parameters to preserve accuracy. This directly reduces training time on compatible GPUs (e.g., NVIDIA V100, A100) without sacrificing model quality, as the loss scaling technique prevents underflow in gradients.

Answer

Switch from GPU instance to CPU instance

Answer

Increase the maximum sequence length

Question 7

A company is building a recommendation system for an e-commerce platform. They have user-item interaction data and want to use matrix factorization. However, the dataset is sparse (99% missing interactions). Which approach should the data scientist take to train the model effectively?

Accepted Answer

Use alternating least squares (ALS) with implicit feedback and assign lower confidence to unobserved interactions. Alternating Least Squares (ALS) with implicit feedback is specifically designed to handle sparse implicit feedback datasets by assigning lower confidence to unobserved interactions (e.g., confidence = 1 + alpha * r_ui, where r_ui is 0 for unobserved). This avoids the pitfalls of treating missing values as zeros (which distorts the factorization) and scales well to 99% sparsity by leveraging weighted regularization.

Answer

Impute missing values with zeros and use singular value decomposition (SVD)

Answer

Remove all users and items with fewer than 10 interactions to reduce sparsity

Answer

Use item-based collaborative filtering with cosine similarity

Question 8

A financial services company is building a model to detect fraudulent credit card transactions. The dataset contains 1 million transactions, with only 0.1% labeled as fraud. The data scientist trains a logistic regression model on the raw dataset and obtains the following results on a held-out test set: accuracy = 99.8%, precision = 50%, recall = 60%, F1 = 0.545. The business requirement is to maximize recall while keeping precision above 80%. Which course of action should the data scientist take to improve the model?

Accepted Answer

Train the model with class weights inversely proportional to class frequencies. Assigning class weights inversely proportional to class frequencies penalizes misclassifications of the minority class (fraud) more heavily during training. This directly addresses the severe class imbalance (0.1% fraud) by forcing the logistic regression model to learn decision boundaries that improve recall, while the weight ratio can be tuned to maintain precision above 80%. Unlike naive resampling, this approach preserves the original data distribution and avoids information loss.

Answer

Use random undersampling of the majority class to balance the dataset

Answer

Collect more historical transaction data and retrain the model

Answer

Apply L2 regularization with a higher penalty to reduce overfitting

Question 9

A data scientist is training a linear regression model to predict house prices. The dataset includes features such as square footage, number of bedrooms, and location. After training, the model achieves an R² of 0.85 on the training set but only 0.60 on the test set. Which of the following is the MOST likely cause of this discrepancy?

Accepted Answer

The model is overfitting the training data. A high R² on the training set (0.85) paired with a significantly lower R² on the test set (0.60) is a classic symptom of overfitting. The model has learned noise and specific patterns in the training data that do not generalize to unseen data, causing poor test performance. Regularization techniques like Lasso or Ridge, or reducing model complexity, would typically address this issue.

Answer

There is multicollinearity among the features

Answer

The model is underfitting the training data

Answer

There is data leakage between the training and test sets

Question 10

A machine learning team is building a multi-class image classifier using a pre-trained ResNet-50 model in Amazon SageMaker. The dataset has 10 classes but is highly imbalanced, with one class representing 80% of the samples. The team wants to improve model performance on the minority classes. Which TWO of the following approaches are most likely to help? (Select TWO.)

Accepted Answer

Oversample the minority classes in the training data.. Oversampling the minority classes (Option A) directly addresses class imbalance by replicating samples from underrepresented classes, giving the model more exposure to them during training. This is a standard data-level technique that helps the ResNet-50 model learn discriminative features for minority classes without altering the loss function or model architecture.

Answer

Reduce the batch size to increase the frequency of weight updates.

Answer

Increase the number of layers in the model.

Answer

Switch to a focal loss function.

Question 11

Drag and drop the steps to deploy a model as a SageMaker endpoint for real-time inference in the correct order.

Accepted Answer

1. Create model, 2. Create endpoint configuration, 3. Create endpoint, 4. Test endpoint. Deployment requires model creation, endpoint configuration, endpoint creation, and testing.

Answer

1. Create endpoint configuration, 2. Create model, 3. Create endpoint, 4. Test endpoint

Answer

1. Create model, 2. Create endpoint, 3. Create endpoint configuration, 4. Test endpoint

Answer

1. Test endpoint, 2. Create model, 3. Create endpoint configuration, 4. Create endpoint

Question 12

Match each hyperparameter tuning strategy to its description.

Accepted Answer

Grid Search: Exhaustively tries all combinations of specified hyperparameter values.. The correct matches are: Grid Search exhaustively searches all combinations; Random Search samples randomly; Bayesian Optimization uses a probabilistic model. Common confusions involve swapping the descriptions of Grid Search and Random Search, or confusing Random Search with Bayesian Optimization.

Answer

Grid Search: Randomly samples hyperparameter combinations from a specified distribution.

Answer

Random Search: Uses a probabilistic model to guide the search towards promising hyperparameters.

Question 13

A machine learning team is deploying a model that performs real-time inference on streaming data from Amazon Kinesis Data Streams. The model requires sub-100ms latency. Which deployment option should the team choose?

Accepted Answer

Deploy the model on an Amazon SageMaker real-time endpoint. A SageMaker real-time endpoint is designed for low-latency, synchronous inference and can consistently deliver sub-100ms responses, making it the correct choice for real-time streaming inference. It keeps the model loaded on persistent instances and responds immediately to each request. This matches the sub-100ms latency requirement for Kinesis-sourced streaming data.

Answer

Use Amazon SageMaker batch transform

Answer

Use Amazon SageMaker asynchronous inference

Answer

Deploy a custom inference container on AWS Lambda

Question 14

A data scientist is training a deep learning model on a large dataset using Amazon SageMaker. The training job is taking too long. The scientist notices that GPU utilization is low and data loading is the bottleneck. Which action should the scientist take to improve training performance?

Accepted Answer

Use Pipe mode for the training data channel. Low GPU utilization with a data loading bottleneck indicates that the CPU cannot feed data to the GPU quickly enough. SageMaker's Pipe mode streams training data directly from S3 without first downloading it to the instance's local storage, reducing I/O overhead and improving data throughput. Option A: Increasing the number of training instances does not address the per-instance data loading bottleneck; each GPU would still be underutilized. Option C: Changing to a CPU instance would be slower because CPUs are less efficient for deep learning training. Option D: Reducing the batch size would decrease GPU utilization further, exacerbating the underutilization problem.

Answer

Increase the number of training instances

Answer

Change the instance type to a CPU instance

Answer

Reduce the batch size

Question 15

A data scientist is building a regression model to predict house prices. The dataset contains many features, some of which are highly correlated. The model is overfitting. Which regularization technique should the scientist use to penalize large coefficients and perform feature selection?

Accepted Answer

L1 regularization (Lasso). L1 regularization (Lasso) adds a penalty equal to the absolute value of the coefficients, which can shrink some coefficients to zero, performing feature selection. L2 regularization (Ridge) penalizes squared coefficients but does not zero them out. Elastic Net combines both. Dropout is for neural networks. Option A (L2 regularization) does not perform feature selection. Option B (L1 regularization) is correct for feature selection. Option C (Elastic Net) combines both but L1 alone is simpler for feature selection. Option D (Dropout) is not applicable to linear regression.

Answer

L2 regularization (Ridge)

Answer

Elastic Net regularization

Answer

Dropout