MLS-C01 Modeling Practice Test 35 — 15 Questions

Question 1

A machine learning team is using Amazon SageMaker to train a linear regression model. The team notices that the training loss decreases rapidly initially but then plateaus at a high value. What is the MOST likely cause?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The learning rate is set too high. A learning rate set too high causes the optimizer to take excessively large steps, overshooting the minimum of the loss function. This results in rapid initial decrease as the model makes large corrections, but then the loss plateaus at a high value because the parameters oscillate around the optimum without converging. In SageMaker's linear regression (typically using stochastic gradient descent), a high learning rate prevents fine-grained convergence, leading to a high plateau.

Answer

The model uses batch normalization

Answer

The learning rate is set too low

Answer

The model is over-regularized with L2 regularization

Question 2

A company is building a binary classification model to predict customer churn. The dataset is highly imbalanced (95% non-churn, 5% churn). The data scientist uses SMOTE to oversample the minority class. After training a logistic regression model, the recall for the churn class is 0.80, but the precision is only 0.10. Which action would MOST likely improve precision without significantly harming recall?

Accepted Answer

Increase the classification threshold for the positive class. Increasing the classification threshold for the positive class makes the model more conservative in predicting churn, which reduces false positives and improves precision. While this may slightly decrease recall (true positives), the trade-off is acceptable given the very low precision. Option A is wrong because random oversampling can lead to overfitting and does not directly address the low precision caused by threshold issues. Option B is wrong because reducing features could discard important information, potentially harming recall without necessarily improving precision. Option D is wrong because decreasing the threshold would further increase false positives, lowering precision even more.

Answer

Use random oversampling instead of SMOTE

Answer

Reduce the number of features in the model

Answer

Decrease the classification threshold for the positive class

Question 3

A data scientist is training a Random Forest model on Amazon SageMaker. The model performs well on the training set but poorly on the test set. Which technique should the data scientist use to address this issue?

Accepted Answer

Decrease the maximum depth of each tree. The model is overfitting, as indicated by high training performance and poor test performance. Decreasing the maximum depth of each tree limits the complexity of individual trees, reducing overfitting by preventing them from memorizing noise in the training data. This is a standard regularization technique for Random Forest models in Amazon SageMaker.

Answer

Increase the number of trees in the forest

Answer

Increase the learning rate

Answer

Increase the maximum depth of each tree

Question 4

A company is using Amazon SageMaker to deploy a model for real-time inference. The model receives requests that are small but arrive in bursts. The data scientist wants to minimize latency and cost. Which deployment option is MOST appropriate?

Accepted Answer

Use Amazon SageMaker Serverless Inference. Amazon SageMaker Serverless Inference is the most appropriate option because it automatically scales compute resources based on request volume, charges only for the compute time used during inference (per-millisecond billing), and has no idle costs. This matches the bursty, small-request pattern perfectly, minimizing both latency and cost without requiring manual instance management.

Answer

Use a real-time endpoint with a single instance

Answer

Use a multi-model endpoint with auto-scaling

Answer

Use a batch transform job triggered by a schedule

Question 5

A data scientist is tuning a gradient boosting model using Amazon SageMaker Automatic Model Tuning. The objective metric is AUC. The training job converges quickly but the final model has low AUC on the validation set. Which hyperparameter should the data scientist adjust to improve validation AUC?

Accepted Answer

Decrease the learning rate and increase the number of rounds. Decreasing the learning rate and increasing the number of rounds is the correct approach because a low learning rate forces the model to take smaller steps toward the optimum, reducing overfitting and allowing more trees to contribute to the ensemble. This combination often improves generalization and validation AUC when the training job converges too quickly, indicating that the model is overfitting or underfitting due to aggressive learning.

Answer

Increase the subsample ratio of training data

Answer

Increase the learning rate

Answer

Increase the maximum depth of trees

Question 6

A data scientist is using Amazon SageMaker to train a deep learning model with a large dataset. The training job fails with a 'CUDA out of memory' error. What is the MOST efficient way to resolve this issue?

Accepted Answer

Reduce the batch size. The 'CUDA out of memory' error occurs when the GPU's memory is insufficient to hold the model parameters, gradients, optimizer states, and the current batch of data. Reducing the batch size decreases the memory footprint per training step, allowing the model to fit within the available GPU memory without requiring a more expensive instance or sacrificing GPU acceleration.

Answer

Switch to a CPU-only instance

Answer

Use a larger instance type with more GPUs

Answer

Increase the batch size

Question 7

A company is building a recommendation system using matrix factorization. The training data contains user-item interactions. The model performs well on the training set but poorly on the test set. Which regularization technique should be applied to improve generalization?

Accepted Answer

Add L2 regularization to the user and item latent factors. L2 regularization (weight decay) penalizes large values in the user and item latent factor matrices, which helps prevent overfitting by encouraging the model to learn smoother, more generalizable representations. This is the standard regularization technique used in matrix factorization for collaborative filtering, as it directly controls the magnitude of the latent vectors without inducing sparsity.

Answer

Add L1 regularization to the user and item latent factors

Answer

Apply dropout to the latent factors during training

Answer

Use batch normalization on the factors

Question 8

A data scientist is training a convolutional neural network (CNN) for image classification using Amazon SageMaker. The training loss decreases steadily but validation loss starts increasing after a few epochs. Which action should the data scientist take to address this issue?

Accepted Answer

Implement early stopping based on validation loss. The described behavior—training loss decreasing while validation loss increases—is a classic sign of overfitting. Early stopping monitors the validation loss and halts training when it stops improving (or starts to increase), preventing the model from memorizing noise in the training data. In SageMaker, this can be implemented using the `EarlyStopping` callback in the framework's estimator or by setting `use_early_stopping` to True in a built-in algorithm.

Answer

Increase the learning rate

Answer

Add more convolutional layers

Answer

Increase the batch size

Question 9

Which TWO actions can help reduce overfitting in a decision tree model? (Choose 2.)

Accepted Answer

Prune the tree after training. Pruning the tree after training removes branches that have little predictive power, reducing overfitting by simplifying the model. This technique directly addresses the variance component of the bias-variance tradeoff, making the model generalize better to unseen data.

Answer

Increase the maximum depth of the tree

Answer

Increase the number of features considered at each split

Answer

Use all training data without validation

Question 10

Which THREE factors should be considered when selecting the appropriate algorithm for a regression problem? (Choose 3.)

Accepted Answer

The number of features relative to the number of samples. The ratio of features to samples directly impacts model complexity and overfitting risk. In high-dimensional settings (e.g., p >> n), algorithms like linear regression may fail due to singular covariance matrices, while regularized methods (Ridge, Lasso) or tree-based models become necessary. This is a core consideration in the bias-variance tradeoff for regression problems.

Answer

The time of day the training will occur

Answer

The color of the data scientist's laptop

Question 11

Which TWO techniques are used for feature scaling? (Choose 2.)

Accepted Answer

Standardization (Z-score normalization). Standardization (Z-score normalization) is a feature scaling technique that transforms data to have a mean of 0 and a standard deviation of 1, using the formula z = (x - μ) / σ. This is essential for algorithms like SVM, k-means, and PCA that assume normally distributed features and are sensitive to feature magnitudes.

Answer

One-hot encoding

Answer

Principal Component Analysis (PCA)

Answer

Label encoding

Question 12

A data scientist is trying to launch a SageMaker training job using an IAM role with the above policy. The training job fails with an access denied error. What is the MOST likely reason?

Accepted Answer

The policy does not include s3:ListBucket permission. The policy grants s3:GetObject and s3:PutObject permissions on the S3 bucket ARN with a /* suffix, but SageMaker training jobs also require s3:ListBucket permission at the bucket level (without the /*) to enumerate objects and validate paths during job creation. Without s3:ListBucket, the training job fails with an access denied error even though read/write permissions are present.

Answer

The sagemaker:StopTrainingJob action is not required

Answer

The sagemaker:CreateTrainingJob action is not allowed for the specific instance type

Answer

The S3 bucket ARN should not include the /* suffix

Question 13

A data scientist runs a SageMaker training job and receives the above error. The S3 bucket 'my-bucket' contains a folder 'data' with a file 'data.csv'. What is the MOST likely cause of the error?

Accepted Answer

The S3 URI should be s3://my-bucket/data/data.csv instead of s3://my-bucket/data. The error occurs because the SageMaker training job expects a specific S3 object URI (pointing to a file), not a prefix (pointing to a folder). When you specify `s3://my-bucket/data`, SageMaker interprets it as a prefix and attempts to list objects under that prefix, but the training channel requires a direct file reference. Using `s3://my-bucket/data/data.csv` provides the exact object path, allowing SageMaker to download the file correctly.

Answer

The instance type ml.m5.large does not have enough memory

Answer

The VolumeSizeInGB is too small to download the data

Answer

The S3 bucket and the training job are in different regions

Question 14

A data scientist is deploying a SageMaker model using CloudFormation. The stack creation fails with the above error. What is the MOST likely cause?

Accepted Answer

The Docker image has not been pushed to the ECR repository. The error indicates that SageMaker cannot find the Docker image specified in the `PrimaryContainer` of the model definition. CloudFormation creates the SageMaker model by referencing an ECR image URI; if that image has not been pushed to the specified ECR repository, the model creation fails immediately. This is the most common cause when the stack creation fails with an error about a missing or inaccessible image.

Answer

The IAM role does not have permissions to access ECR

Answer

The model name is incorrect

Answer

The instance type specified in the endpoint configuration is not available

Question 15

A data scientist is training a binary classification model on a highly imbalanced dataset (0.1% positive class). To improve recall, the team decides to use SageMaker's built-in XGBoost algorithm. Which parameter adjustment is most likely to increase recall without significantly sacrificing precision?

Accepted Answer

Set scale_pos_weight to the ratio of negative to positive samples. Setting scale_pos_weight to the ratio of negative to positive samples (approximately 999:1) tells XGBoost to assign a higher penalty to misclassifications of the minority positive class. This directly increases the gradient contribution from positive samples during training, which shifts the decision boundary to improve recall while maintaining a balance that avoids excessive false positives, thus preserving precision.

Answer

Increase max_depth from 5 to 10

Answer

Reduce num_round from 100 to 50

Answer

Increase subsample from 0.8 to 1.0