MLS-C01 Modeling Practice Test 28 — 15 Questions

Question 1

A machine learning team is deploying a model using Amazon SageMaker. The model receives requests with sparse high-dimensional features. The team wants to minimize inference latency. Which SageMaker endpoint configuration is MOST suitable?

Accepted Answer

Use a multi-model endpoint on a GPU instance. Option D is correct because multi-model endpoints on GPU instances allow multiple models to be loaded into memory on a single GPU-backed instance, reducing cold-start latency for sparse high-dimensional features by keeping models warm and leveraging GPU parallelism for inference. This minimizes inference latency compared to other configurations by avoiding the overhead of separate endpoint invocations and optimizing resource utilization for high-dimensional data.

Answer

Use a multi-variant endpoint with two variants

Answer

Use a serverless endpoint with provisioned concurrency

Answer

Use a single model endpoint with a large instance type

Question 2

A data scientist is training a linear regression model and observes that the training loss is low but validation loss is high. Which step should the data scientist take to address this issue?

Accepted Answer

Apply L2 regularization to the model. Option D is correct because high validation loss indicates overfitting; regularization reduces overfitting. Option A is wrong because adding more features may increase overfitting. Option B is wrong because increasing training time typically increases overfitting. Option C is wrong because reducing training data may worsen overfitting.

Answer

Increase the number of training epochs

Answer

Reduce the size of the training dataset

Answer

Add more features to the model

Question 3

A company uses Amazon SageMaker to train a deep learning model on a GPU instance. The training job is taking too long. Which action would MOST likely reduce training time?

Accepted Answer

Use a larger GPU instance type, such as p3.16xlarge. Option C is correct because using a larger GPU instance like p3.16xlarge provides significantly more GPU memory, CUDA cores, and memory bandwidth, which allows for larger batch sizes and more efficient parallel processing of matrix operations. This directly reduces training time for deep learning models by enabling faster forward and backward passes through the network, especially when the model is large enough to fully utilize the additional GPU resources.

Answer

Reduce the mini-batch size

Answer

Use distributed data parallelism across multiple smaller instances

Answer

Reduce the number of epochs

Question 4

A data scientist is building a binary classifier for loan default prediction. The cost of a false negative (missing a default) is 10 times higher than the cost of a false positive. Which evaluation metric is MOST appropriate?

Accepted Answer

F-beta score with beta=2. The F-beta score with beta=2 is the most appropriate metric because it weights recall (sensitivity) higher than precision, which is critical when false negatives are 10 times more costly than false positives. Beta=2 means recall is considered 2^2 = 4 times more important than precision, directly aligning with the asymmetric cost structure. This allows the model to be tuned to minimize missed defaults, even at the expense of more false alarms.

Answer

Precision

Answer

Accuracy

Answer

Area under the ROC curve

Question 5

A data scientist is training a random forest model. During hyperparameter tuning, which parameter is MOST effective at reducing overfitting?

Accepted Answer

Decrease the maximum depth of each tree. Decreasing the maximum depth of each tree limits the complexity of individual trees, preventing them from memorizing noise and outliers in the training data. This directly reduces overfitting by enforcing simpler decision boundaries, which is a core regularization technique for ensemble methods like Random Forest.

Answer

Increase the number of trees

Answer

Increase the number of features considered per split

Answer

Increase the maximum depth of each tree

Question 6

A company is using Amazon SageMaker to train a model. The training job is using a large dataset stored in S3. The data scientist notices that the training job is spending a significant amount of time reading data from S3. Which approach would BEST reduce data loading time?

Accepted Answer

Use the Pipe mode input for the training data. Pipe mode streams data directly from S3 into the training algorithm without first downloading it to the training instance's local storage. This eliminates the I/O bottleneck of writing large datasets to disk, significantly reducing data loading time compared to File mode, which downloads the entire dataset before training begins.

Answer

Use the File mode input with a larger instance

Answer

Use a larger training instance with more CPU

Answer

Increase the batch size to reduce the number of batches

Question 7

A data scientist is using Amazon SageMaker Autopilot to automatically build a model. The dataset contains a mix of numerical and categorical features. After the experiment completes, Autopilot provides several candidate pipelines. Which pipeline is MOST likely to be ranked highest by Autopilot?

Accepted Answer

The pipeline with the lowest validation loss. Amazon SageMaker Autopilot ranks candidate pipelines by their objective metric on the validation dataset, which is typically the validation loss (e.g., cross-entropy for classification or mean squared error for regression). The pipeline with the lowest validation loss generalizes best to unseen data, making it the highest-ranked candidate. Autopilot uses hold-out validation or cross-validation to compute this metric, ensuring the ranking reflects out-of-sample performance rather than overfitting to the training set.

Answer

The pipeline with the simplest model (e.g., linear classifier)

Answer

The pipeline with the fastest training time

Answer

The pipeline with the lowest training loss

Question 8

A data scientist is training a neural network for image classification. The training loss is decreasing steadily, but the validation loss starts increasing after a few epochs. What is the MOST likely cause?

Accepted Answer

The model is overfitting to the training data. The correct answer is D because the validation loss increasing while the training loss continues to decrease is the classic signature of overfitting. The model is memorizing the training data (including noise) rather than learning generalizable patterns, causing it to perform poorly on unseen validation data.

Answer

The learning rate is too high

Answer

The gradients are vanishing

Answer

The model is underfitting

Question 9

Which TWO approaches are valid for handling missing categorical values in a dataset before training a machine learning model?

Accepted Answer

Impute missing values with the mode of the column. Option B is correct because the mode (most frequent value) is the only valid measure of central tendency for categorical data, as it identifies the most common category. Imputing with the mode preserves the distribution of categories and is a standard technique for handling missing categorical values in preprocessing pipelines like scikit-learn's SimpleImputer with strategy='most_frequent'.

Answer

Remove all rows with missing values

Answer

Impute missing values with the median of the column

Answer

Impute missing values with the mean of the column

Question 10

Which THREE factors should be considered when choosing between a parametric and a non-parametric machine learning model?

Accepted Answer

Non-parametric models are generally more flexible. Option A is correct because non-parametric models, such as k-nearest neighbors or decision trees, do not assume a fixed functional form for the data, allowing them to capture complex, non-linear relationships. This flexibility makes them well-suited for datasets where the underlying distribution is unknown or highly irregular, but it also increases the risk of overfitting if not properly regularized.

Answer

Parametric models have lower bias than non-parametric models

Answer

Non-parametric models are less prone to overfitting

Question 11

Which TWO techniques can help reduce overfitting in a decision tree model?

Accepted Answer

Limit the maximum depth of the tree. Limiting the maximum depth of the tree (Option C) directly restricts the number of splits, preventing the model from learning overly specific patterns in the training data. Pruning the tree after training (Option D) removes branches that have little predictive power, reducing variance and improving generalization. Both techniques combat overfitting by controlling the complexity of the decision tree.

Answer

Increase the number of trees in the forest

Answer

Increase the number of features considered per split

Answer

Increase the maximum depth of the tree

Question 12

A data scientist is training a binary classification model on an imbalanced dataset where the positive class accounts for 5% of the data. The model achieves 95% accuracy but has a recall of only 10% for the positive class. Which metric should the data scientist primarily use to evaluate model performance?

Accepted Answer

F1 Score. The F1 Score is the harmonic mean of precision and recall, making it ideal for imbalanced datasets where accuracy is misleading. With 95% accuracy but only 10% recall, the model is simply predicting the majority class (negative) almost always, so F1 Score captures the trade-off between false positives and false negatives better than accuracy or AUC-ROC.

Answer

RMSE

Answer

Accuracy

Answer

AUC-ROC

Question 13

A company uses Amazon SageMaker to train a linear regression model. After training, the model shows high bias on the training set. Which action is MOST likely to reduce bias?

Accepted Answer

Add more features. High bias indicates that the model is underfitting the training data, meaning it is too simple to capture the underlying patterns. Adding more features increases the model's capacity to learn complex relationships, directly addressing underfitting by reducing bias. In SageMaker, this can be done by engineering additional input columns or using feature transformations before training.

Answer

Collect more training data

Answer

Apply L2 regularization

Answer

Deploy the model to a larger instance

Question 14

A machine learning team is building a recommendation system for an e-commerce platform. They have user-item interaction data (clicks, purchases). They need to choose an algorithm that can capture both user and item latent factors and handle missing data. Which algorithm should they use?

Accepted Answer

Matrix factorization. Matrix factorization is the correct choice because it decomposes the user-item interaction matrix into lower-dimensional latent factors for users and items, capturing underlying patterns in preferences. It naturally handles missing data by learning from observed interactions only, making it ideal for recommendation systems with sparse data.

Answer

Linear regression

Answer

Principal component analysis (PCA)

Answer

Convolutional neural network (CNN)

Question 15

A data scientist is training a decision tree classifier and notices that the model performs well on training data but poorly on test data. Which technique should the data scientist use to address this issue?

Accepted Answer

Prune the tree. Pruning the tree reduces overfitting by removing branches that have little statistical significance or that capture noise in the training data. This technique improves generalization to unseen test data, which directly addresses the symptom of high training accuracy and low test accuracy.

Answer

Use a different split criterion

Answer

Apply L1 regularization

Answer

Increase tree depth