MLS-C01 Timed Practice Test 5 — 10 Questions

Question 1

Which TWO metrics are MOST appropriate for evaluating a regression model that predicts house prices, where the business is most sensitive to large errors?

Accepted Answer

Root Mean Squared Error (RMSE). RMSE is most appropriate because it squares the errors before averaging, which heavily penalizes large errors. Since the business is most sensitive to large errors in house price predictions, RMSE directly aligns with this requirement by amplifying the impact of outliers, making it a suitable metric for evaluating model performance in this context.

Answer

Accuracy

Answer

Mean Absolute Error (MAE)

Answer

R-squared

Question 2

Which THREE techniques can help reduce overfitting in a neural network trained on a small dataset?

Accepted Answer

Apply L2 weight regularization. L2 weight regularization (also known as weight decay) penalizes large weights by adding a term to the loss function proportional to the sum of squared weights. This forces the network to learn simpler patterns and reduces sensitivity to noise in the training data, which is especially helpful when the dataset is small and prone to overfitting.

Answer

Increase the number of hidden layers

Answer

Train for more epochs

Question 3

A data scientist is training a neural network for image classification. The training loss is not decreasing significantly, and the validation loss is high. Which TWO actions should the scientist take to address potential vanishing gradients?

Accepted Answer

Use ReLU activation functions in hidden layers. ReLU activation functions help mitigate vanishing gradients because they output a constant gradient of 1 for positive inputs, preventing the gradient from shrinking as it propagates backward through many layers. This avoids the exponential decay of gradients that occurs with saturating activations like sigmoid or tanh, enabling effective training of deep networks.

Answer

Increase the learning rate

Answer

Switch activation functions from ReLU to sigmoid

Answer

Remove dropout layers

Question 4

A company is deploying a real-time fraud detection system using a gradient boosting model on AWS SageMaker. The model uses 200 features and is trained on 50 GB of data. The inference latency requirement is under 10 ms per request. During load testing, the endpoint shows average latency of 15 ms. Which change is MOST likely to reduce latency below 10 ms?

Accepted Answer

Reduce the number of features to the top 50 based on feature importance. Reducing the number of features from 200 to the top 50 directly decreases the amount of data each inference request must process, which lowers both feature engineering overhead and model evaluation time. For gradient boosting models on SageMaker, fewer features mean fewer decision tree splits to traverse per prediction, which can significantly reduce latency without requiring hardware changes. This is the most direct and cost-effective way to meet the 10 ms requirement.

Answer

Switch to a GPU-based instance type

Answer

Increase the number of trees in the model

Answer

Use a larger batch size for inference

Question 5

A data scientist is training a binary classifier to predict customer churn. The dataset has 10,000 samples, with 500 churners (positive class). The scientist trains a logistic regression model and obtains an F1-score of 0.6. To improve the F1-score, which approach is MOST likely to be effective?

Accepted Answer

Apply SMOTE to oversample the minority class. The dataset is highly imbalanced (500 churners out of 10,000 samples, a 5% positive rate). Logistic regression trained on such imbalance tends to bias toward the majority class, resulting in low recall for the minority class and a poor F1-score. SMOTE (Synthetic Minority Oversampling Technique) generates synthetic samples for the minority class by interpolating between existing minority instances, which balances the class distribution and allows the model to learn a better decision boundary, directly improving recall and F1-score.

Answer

Increase the regularization strength (C)

Answer

Apply PCA to reduce feature dimensionality

Answer

Use the original dataset without any modification

Question 6

Drag and drop the steps to train a model using Amazon SageMaker built-in algorithm in the correct order.

Question 7

Drag and drop the steps to deploy a model as a SageMaker endpoint for real-time inference in the correct order.

Question 8

Drag and drop the steps to create a data processing job using Amazon SageMaker Processing in the correct order.

Question 9

A company is using Amazon SageMaker to train a large language model. The training job is taking too long. The data scientist wants to reduce training time without sacrificing model accuracy. Which THREE strategies are MOST appropriate?

Accepted Answer

Use mixed precision training (float16). Mixed precision training (float16) reduces memory usage and accelerates computation by using half-precision floating-point numbers for most operations, while maintaining a single-precision copy of critical parameters to preserve accuracy. This directly reduces training time on compatible GPUs (e.g., NVIDIA V100, A100) without sacrificing model quality, as the loss scaling technique prevents underflow in gradients.

Answer

Switch from GPU instance to CPU instance

Answer

Increase the maximum sequence length

Question 10

A data scientist is training a binary classification model on a dataset with a severe class imbalance (95% negative, 5% positive). The model achieves 95% accuracy but only correctly identifies 10% of the positive class. Which metric should the data scientist use to evaluate model performance?

Accepted Answer

F1 score. The F1 score is the harmonic mean of precision and recall, making it robust to class imbalance. With 95% accuracy but only 10% recall on the positive class, the model is essentially a trivial classifier that predicts the majority class. F1 score captures both false positives and false negatives, providing a balanced view of performance on the minority class.

Answer

Log loss

Answer

Accuracy

Answer

Area under the ROC curve (AUC)