MLS-C01 Modeling Practice Test 29 — 15 Questions

Question 1

Which TWO techniques can help reduce overfitting in a decision tree model?

Accepted Answer

Limit the maximum depth of the tree. Limiting the maximum depth of the tree (Option C) directly restricts the number of splits, preventing the model from learning overly specific patterns in the training data. Pruning the tree after training (Option D) removes branches that have little predictive power, reducing variance and improving generalization. Both techniques combat overfitting by controlling the complexity of the decision tree.

Answer

Increase the number of trees in the forest

Answer

Increase the number of features considered per split

Answer

Increase the maximum depth of the tree

Question 2

A data scientist is training a binary classification model on an imbalanced dataset where the positive class accounts for 5% of the data. The model achieves 95% accuracy but has a recall of only 10% for the positive class. Which metric should the data scientist primarily use to evaluate model performance?

Accepted Answer

F1 Score. The F1 Score is the harmonic mean of precision and recall, making it ideal for imbalanced datasets where accuracy is misleading. With 95% accuracy but only 10% recall, the model is simply predicting the majority class (negative) almost always, so F1 Score captures the trade-off between false positives and false negatives better than accuracy or AUC-ROC.

Answer

RMSE

Answer

Accuracy

Answer

AUC-ROC

Question 3

A company uses Amazon SageMaker to train a linear regression model. After training, the model shows high bias on the training set. Which action is MOST likely to reduce bias?

Accepted Answer

Add more features. High bias indicates that the model is underfitting the training data, meaning it is too simple to capture the underlying patterns. Adding more features increases the model's capacity to learn complex relationships, directly addressing underfitting by reducing bias. In SageMaker, this can be done by engineering additional input columns or using feature transformations before training.

Answer

Collect more training data

Answer

Apply L2 regularization

Answer

Deploy the model to a larger instance

Question 4

A machine learning team is building a recommendation system for an e-commerce platform. They have user-item interaction data (clicks, purchases). They need to choose an algorithm that can capture both user and item latent factors and handle missing data. Which algorithm should they use?

Accepted Answer

Matrix factorization. Matrix factorization is the correct choice because it decomposes the user-item interaction matrix into lower-dimensional latent factors for users and items, capturing underlying patterns in preferences. It naturally handles missing data by learning from observed interactions only, making it ideal for recommendation systems with sparse data.

Answer

Linear regression

Answer

Principal component analysis (PCA)

Answer

Convolutional neural network (CNN)

Question 5

A data scientist is training a decision tree classifier and notices that the model performs well on training data but poorly on test data. Which technique should the data scientist use to address this issue?

Accepted Answer

Prune the tree. Pruning the tree reduces overfitting by removing branches that have little statistical significance or that capture noise in the training data. This technique improves generalization to unseen test data, which directly addresses the symptom of high training accuracy and low test accuracy.

Answer

Use a different split criterion

Answer

Apply L1 regularization

Answer

Increase tree depth

Question 6

A company uses Amazon SageMaker to train an XGBoost model on a large dataset. Training takes a long time. Which action can reduce training time without significantly affecting model accuracy?

Accepted Answer

Enable early stopping. Early stopping halts training when the model's performance on a validation set stops improving for a specified number of rounds. This prevents overfitting and reduces training time by eliminating unnecessary iterations, while typically preserving accuracy because the optimal model is already found.

Answer

Use a deep neural network instead

Answer

Increase the learning rate

Answer

Use a larger instance type

Question 7

A data scientist is building a model to predict customer churn. The dataset has 20 features, including categorical variables with high cardinality (e.g., ZIP code). The data scientist wants to use a linear model. Which feature engineering technique is MOST appropriate for the high-cardinality categorical features?

Accepted Answer

Target encoding. Target encoding is the most appropriate technique for high-cardinality categorical features when using a linear model because it replaces each category with the mean of the target variable for that category, creating a numeric feature that captures the relationship between the category and the target without exploding the feature space. One-hot encoding would create an unmanageable number of binary columns (e.g., thousands of ZIP codes), leading to the curse of dimensionality and making the linear model unstable or computationally infeasible.

Answer

One-hot encoding

Answer

TF-IDF

Answer

Standard scaling

Question 8

A machine learning engineer needs to deploy a model that requires low latency (under 10 ms) for real-time inference. The model is a small ensemble of decision trees. Which Amazon SageMaker endpoint configuration is MOST appropriate?

Accepted Answer

Real-time endpoint. Real-time endpoints in Amazon SageMaker are designed for low-latency inference (typically under 10 ms) and are the correct choice for deploying a small ensemble of decision trees that needs to respond to individual prediction requests in real time. They keep the model loaded and ready, providing a persistent HTTPS endpoint that can serve predictions with minimal overhead.

Answer

Batch transform

Answer

Training job

Answer

Multi-model endpoint

Question 9

A data scientist is training a deep learning model using TensorFlow on Amazon SageMaker. The training job uses a single GPU instance but the GPU utilization is low. Which action is MOST likely to improve GPU utilization?

Accepted Answer

Increase the batch size. Increasing the batch size allows the GPU to process more data in parallel per training step, which keeps the GPU compute units busier and reduces idle time. In TensorFlow on SageMaker, a small batch size can cause the GPU to finish computation quickly and then wait for the next batch to be loaded, leading to low utilization. This is the most direct way to improve GPU throughput without changing the instance or model architecture.

Answer

Use a smaller instance type

Answer

Add more features

Answer

Decrease the number of epochs

Question 10

A company uses Amazon SageMaker to train a model for fraud detection. The training data is highly imbalanced. The data scientist uses SMOTE to oversample the minority class. However, the model still has poor recall on the minority class. Which additional technique should the data scientist consider?

Accepted Answer

Use class weights in the loss function. SMOTE generates synthetic samples for the minority class, but it does not directly address the model's tendency to prioritize the majority class during training. By assigning higher class weights to the minority class in the loss function, the model penalizes misclassifications of minority samples more heavily, which directly improves recall on that class. This technique is especially effective when combined with oversampling, as it forces the optimizer to focus on the underrepresented class during gradient updates.

Answer

One-vs-rest encoding

Answer

L1 regularization

Answer

Principal component analysis (PCA)

Question 11

Which TWO metrics are appropriate for evaluating a binary classification model trained on imbalanced data? (Select TWO.)

Accepted Answer

F1 score. The F1 score is appropriate for imbalanced binary classification because it balances precision and recall, making it robust when the positive class is rare. Unlike accuracy, it does not get inflated by a majority negative class, and it directly penalizes models that predict the majority class for all instances.

Answer

Log loss

Answer

Accuracy

Answer

ROC-AUC

Question 12

Which THREE techniques can help reduce overfitting in a neural network? (Select THREE.)

Accepted Answer

Dropout. Dropout is correct because it randomly deactivates a fraction of neurons during training, forcing the network to learn redundant representations and preventing co-adaptation of features. This reduces overfitting by acting as an ensemble method without increasing computational cost at inference time.

Answer

Increase training epochs

Answer

Increase the number of layers

Question 13

Which TWO actions are valid ways to handle missing data in a dataset before training a machine learning model? (Select TWO.)

Accepted Answer

Delete rows with missing values. Deleting rows with missing values (listwise deletion) is a straightforward and valid approach when the missing data is random and the dataset is large enough that the loss of rows does not significantly reduce statistical power or introduce bias. This method avoids the need to estimate missing values and is commonly used in practice when the proportion of missing data is low.

Answer

Remove all features that have any missing values

Answer

Replace missing values with the maximum value

Answer

Ignore missing values and train the model

Question 14

Refer to the exhibit. An IAM policy is attached to a SageMaker notebook instance role. When the data scientist tries to run a training job that writes model artifacts to 's3://my-bucket/models/', the job fails with an access denied error. What is the MOST likely cause?

Accepted Answer

Missing s3:PutObject permission for the output S3 bucket. The error occurs because the IAM policy attached to the SageMaker notebook instance role does not grant the s3:PutObject permission on the 's3://my-bucket/models/' path. SageMaker training jobs require this permission to write model artifacts to the specified S3 output bucket. Without it, the API call to upload the model fails with an access denied error, even if other S3 actions are allowed.

Answer

The IAM role does not have a trust policy

Answer

The policy does not include any S3 actions

Answer

The sagemaker:CreateTrainingJob action is not allowed on the specific resource

Question 15

Refer to the exhibit. A SageMaker training job using the built-in Linear Learner algorithm fails with 'Loss function returned NaN'. Which hyperparameter change is MOST likely to resolve this issue?

Accepted Answer

Reduce learning_rate to 0.01. The 'Loss function returned NaN' error in SageMaker's built-in Linear Learner algorithm typically occurs when the learning rate is too high, causing gradient updates to overshoot optimal parameters and diverge. Reducing the learning rate to 0.01 stabilizes training by ensuring smaller, more controlled weight updates, preventing numerical instability that leads to NaN loss.

Answer

Increase the learning rate to 0.5

Answer

Increase mini_batch_size to 2000

Answer

Decrease epochs to 5