MLS-C01 Modeling Practice Test 37 — 15 Questions

Question 1

A company is building a recommendation system for an e-commerce platform. The data includes user IDs and item IDs. Which SageMaker built-in algorithm is most appropriate?

Accepted Answer

Factorization Machines. Factorization Machines (FM) are specifically designed for recommendation tasks with sparse, high-dimensional categorical data like user IDs and item IDs. They model pairwise interactions between features (e.g., user-item interactions) using factorized parameters, making them highly effective for collaborative filtering and implicit feedback scenarios in e-commerce.

Answer

BlazingText

Answer

XGBoost

Answer

Image Classification

Question 2

A company is using a custom Docker container in SageMaker for training. The training job fails with 'ResourceLimitExceeded' error. Which action should the data scientist take?

Accepted Answer

Request a limit increase for the instance type. The 'ResourceLimitExceeded' error in SageMaker indicates that the AWS account has reached a service quota for the specified instance type (e.g., ml.p3.2xlarge). This is a quota limit, not a performance or resource exhaustion issue within the training job itself. The correct action is to request a limit increase via the AWS Service Quotas console or AWS Support, which raises the maximum number of concurrent instances or total vCPUs allowed for that instance family.

Answer

Use a smaller instance type

Answer

Reduce the number of epochs

Answer

Use a pre-built SageMaker container instead

Question 3

A data scientist needs to evaluate a binary classification model's performance. Which metric is most appropriate when the cost of false positives is very high?

Accepted Answer

Precision. Precision is the most appropriate metric when the cost of false positives is very high because it measures the proportion of positive identifications that were actually correct. In binary classification, precision = TP / (TP + FP), so a high precision means very few false positives occur, directly minimizing the costly error type.

Answer

Accuracy

Answer

F1 score

Answer

Recall

Question 4

A data scientist is building a binary classifier and wants to evaluate model performance. Which THREE metrics are most commonly used?

Accepted Answer

Precision. For a binary classifier, Precision (C) is correct because it measures the proportion of positive predictions that are truly positive (TP / (TP + FP)), which is essential when false positives matter. Recall (D) is correct because it measures the proportion of actual positives correctly identified (TP / (TP + FN)), which is essential when false negatives matter. Accuracy (E) is correct because it measures the overall fraction of correct predictions ((TP + TN) / total), a standard baseline metric for classification. Mean Absolute Error (A) and RMSE (B) are not correct here because they are regression metrics that quantify the average magnitude of continuous prediction errors, not classification performance.

Answer

Mean Absolute Error

Answer

RMSE

Question 5

Refer to the exhibit. What is the recall of the model?

Accepted Answer

0.80. Recall is calculated as True Positives divided by the sum of True Positives and False Negatives. From the confusion matrix, True Positives = 80 and False Negatives = 20, so recall = 80 / (80 + 20) = 0.80. Option B is correct.

Answer

0.85

Answer

0.89

Answer

0.90

Question 6

A data scientist is training a classification model on an imbalanced dataset where the positive class represents only 5% of the data. Which technique would BEST address the class imbalance without discarding data?

Accepted Answer

Use SMOTE to generate synthetic samples for the minority class. SMOTE (Synthetic Minority Oversampling Technique) is the best choice because it generates synthetic samples for the minority class by interpolating between existing minority instances, effectively balancing the dataset without discarding any data. This avoids the information loss of undersampling and the overfitting risk of simple random oversampling, making it ideal for a 5% positive class scenario.

Answer

Randomly undersample the majority class

Answer

Adjust the decision threshold to 0.95

Answer

Randomly oversample the minority class with replacement

Question 7

A data scientist is training a linear regression model and observes that the training loss is low but validation loss is high. Which step should the data scientist take to address this issue?

Accepted Answer

Apply L2 regularization to the model. Low training loss with high validation loss is the classic signature of overfitting. L2 regularization (ridge) adds a penalty term proportional to the square of the weights to the loss function, which constrains model complexity and reduces variance, thereby improving validation performance. This is the standard first-line remedy for overfitting in linear regression.

Answer

Increase the number of training epochs

Answer

Reduce the size of the training dataset

Answer

Add more features to the model

Question 8

A data scientist is building a binary classifier for loan default prediction. The cost of a false negative (missing a default) is 10 times higher than the cost of a false positive. Which evaluation metric is MOST appropriate?

Accepted Answer

F-beta score with beta=2. The F-beta score with beta=2 is the most appropriate metric because it weights recall (sensitivity) higher than precision, which is critical when false negatives are 10 times more costly than false positives. Beta=2 means recall is considered 2^2 = 4 times more important than precision, directly aligning with the asymmetric cost structure. This allows the model to be tuned to minimize missed defaults, even at the expense of more false alarms.

Answer

Precision

Answer

Accuracy

Answer

Area under the ROC curve

Question 9

A data scientist is training a random forest model. During hyperparameter tuning, which parameter is MOST effective at reducing overfitting?

Accepted Answer

Decrease the maximum depth of each tree. Decreasing the maximum depth of each tree limits the complexity of individual trees, preventing them from memorizing noise and outliers in the training data. This directly reduces overfitting by enforcing simpler decision boundaries, which is a core regularization technique for ensemble methods like Random Forest.

Answer

Increase the number of trees

Answer

Increase the number of features considered per split

Answer

Increase the maximum depth of each tree

Question 10

A company is using Amazon SageMaker to train a model. The training job is using a large dataset stored in S3. The data scientist notices that the training job is spending a significant amount of time reading data from S3. Which approach would BEST reduce data loading time?

Accepted Answer

Use the Pipe mode input for the training data. Pipe mode streams data directly from S3 into the training algorithm without first downloading it to the training instance's local storage. This eliminates the I/O bottleneck of writing large datasets to disk, significantly reducing data loading time compared to File mode, which downloads the entire dataset before training begins.

Answer

Use the File mode input with a larger instance

Answer

Use a larger training instance with more CPU

Answer

Increase the batch size to reduce the number of batches

Question 11

A data scientist is using Amazon SageMaker Autopilot to automatically build a model. The dataset contains a mix of numerical and categorical features. After the experiment completes, Autopilot provides several candidate pipelines. Which pipeline is MOST likely to be ranked highest by Autopilot?

Accepted Answer

The pipeline with the lowest validation loss. Amazon SageMaker Autopilot ranks candidate pipelines by their objective metric on the validation dataset, which is typically the validation loss (e.g., cross-entropy for classification or mean squared error for regression). The pipeline with the lowest validation loss generalizes best to unseen data, making it the highest-ranked candidate. Autopilot uses hold-out validation or cross-validation to compute this metric, ensuring the ranking reflects out-of-sample performance rather than overfitting to the training set.

Answer

The pipeline with the simplest model (e.g., linear classifier)

Answer

The pipeline with the fastest training time

Answer

The pipeline with the lowest training loss

Question 12

A data scientist is training a neural network for image classification. The training loss is decreasing steadily, but the validation loss starts increasing after a few epochs. What is the MOST likely cause?

Accepted Answer

The model is overfitting to the training data. The validation loss increasing while the training loss continues to decrease is the classic signature of overfitting. The model is memorizing the training data (including noise) rather than learning generalizable patterns, causing it to perform poorly on unseen validation data.

Answer

The learning rate is too high

Answer

The gradients are vanishing

Answer

The model is underfitting

Question 13

Which THREE factors should be considered when choosing between a parametric and a non-parametric machine learning model?

Accepted Answer

Non-parametric models are generally more flexible. Option A is correct because non-parametric models, such as k-NN, decision trees, or kernel SVMs, do not assume a fixed functional form and can adapt their complexity to the data, making them generally more flexible. Option C is correct because parametric models like linear or logistic regression estimate a fixed, small set of parameters, so training is typically faster than for non-parametric models that must retain or process much of the training data. Option E is correct because parametric models have strong structural assumptions and a fixed number of parameters, so they usually need less training data to estimate those parameters reliably. Option B is incorrect because non-parametric models typically have lower bias and higher variance, not the reverse. Option D is incorrect because non-parametric models are generally more prone to overfitting, not less, due to their higher flexibility.

Answer

Parametric models have lower bias than non-parametric models

Answer

Non-parametric models are less prone to overfitting

Question 14

Which TWO techniques can help reduce overfitting in a decision tree model?

Accepted Answer

Limit the maximum depth of the tree. Limiting the maximum depth of the tree (option C) is correct because it constrains how finely the model can partition the feature space, preventing the tree from growing deep enough to memorize noise in the training data. Pruning the tree after training (option D) is correct because post-pruning removes branches that provide little predictive power on validation data, reducing variance and thus overfitting. Increasing the number of trees in the forest (option A) applies to ensemble methods like Random Forest and does not reduce overfitting in a single decision tree. Increasing the number of features considered per split (option B) tends to make the tree more complex and can worsen overfitting. Increasing the maximum depth of the tree (option E) directly allows more splits and greater model complexity, which increases overfitting rather than reducing it.

Answer

Increase the number of trees in the forest

Answer

Increase the number of features considered per split

Answer

Increase the maximum depth of the tree

Question 15

A data scientist is training a binary classification model on an imbalanced dataset where the positive class accounts for 5% of the data. The model achieves 95% accuracy but has a recall of only 10% for the positive class. Which metric should the data scientist primarily use to evaluate model performance?

Accepted Answer

F1 Score. The F1 Score is the harmonic mean of precision and recall, making it ideal for imbalanced datasets where accuracy is misleading. With 95% accuracy but only 10% recall, the model is simply predicting the majority class (negative) almost always, so F1 Score captures the trade-off between false positives and false negatives better than accuracy or AUC-ROC.

Answer

RMSE

Answer

Accuracy

Answer

AUC-ROC