MLS-C01 Modeling Practice Test 16 — 15 Questions

Question 1

A company is using Amazon SageMaker to deploy a model for real-time inference. The model has a latency requirement of less than 100 milliseconds. During testing, the latency is around 150 milliseconds. Which action can most likely reduce the latency to meet the requirement?

Accepted Answer

Use a larger instance type for the endpoint.. Enabling data capture adds overhead and increases latency. Using a larger instance type would provide more compute and reduce latency, but may increase cost. Reducing the batch size for inference (if batching is used) can reduce latency because the model processes fewer requests at once. However, the question implies a real-time endpoint which typically processes one request at a time; batch size might be 1. Increasing the variant weight for the production variant is for traffic routing, not latency. The most direct is to use a more powerful instance type. But also consider that increasing batch size (if using multi-record) increases latency. Reducing batch size reduces latency. However, for a real-time endpoint, the instance type is key. I'll go with using a larger instance type.

Answer

Reduce the batch size for inference.

Answer

Enable data capture for the endpoint.

Answer

Increase the initial variant weight for the production variant.

Question 2

A data scientist is using Amazon SageMaker to train a classification model. The dataset contains categorical features with high cardinality. Which encoding method is most appropriate for handling high-cardinality categorical features in a linear model?

Accepted Answer

Target encoding. One-hot encoding creates many binary columns, which can cause the curse of dimensionality for high-cardinality features. Label encoding assigns arbitrary integers, which linear models may interpret as ordinal. Target encoding (mean encoding) replaces categories with the mean of the target variable, which captures information without expanding dimensionality. This is often used for high-cardinality features. Ordinal encoding is similar to label encoding.

Answer

Label encoding

Answer

One-hot encoding

Answer

Ordinal encoding

Question 3

A data scientist is using Amazon SageMaker to train a random forest model for a binary classification task. The dataset has 50 features and 10,000 samples. The model achieves high training accuracy but poor test accuracy. Which TWO actions should the scientist take to improve generalization?

Accepted Answer

Reduce the max_depth of the trees.. The model is overfitting. Increasing max_depth would increase overfitting. Reducing the number of trees (n_estimators) generally increases bias and may reduce overfitting, but random forest is robust to overfitting with more trees. Typically, increasing trees reduces overfitting. However, reducing tree depth (max_depth) reduces model complexity. Also, increasing the minimum samples per leaf reduces overfitting. So correct actions are: reduce max_depth and increase min_samples_leaf. Option B (increase n_estimators) can also help, but it is not a direct fix for overfitting; random forest with more trees tends to generalize better. But the question says 'which TWO'. I'll go with A and D as they directly reduce complexity. However, increasing n_estimators is also a common practice to reduce overfitting. Let's check: In random forest, more trees reduce variance and overfitting. So B is also correct. But we need exactly 2 correct. The question says 'Which TWO'. I need to provide two correct options. I'll choose A and D. But I'll reconsider: Reducing max_depth (A) and increasing min_samples_leaf (D) are standard ways to reduce overfitting. Increasing n_estimators (B) also helps but may increase training time. The question likely expects A and D. I'll set A and D as correct.

Answer

Increase the max_samples parameter.

Answer

Increase the max_features parameter.

Answer

Increase the number of trees (n_estimators).

Question 4

A company is using Amazon SageMaker to train an XGBoost model. The training data contains missing values. Which TWO methods can XGBoost handle missing values internally?

Accepted Answer

Learn the best direction to go when a value is missing.. XGBoost can handle missing values by learning the best direction to go when a value is missing (sparsity-aware algorithm). You don't need to impute or drop. However, the question asks 'which TWO methods can XGBoost handle missing values internally?' The correct answer is that XGBoost can treat missing values as a separate category and learn the optimal split direction. Also, you can set a default direction. But the options: A (impute with mean) is not internal; B (drop rows) is not; C (treat missing as a separate category) is correct; D (learn best split direction) is correct; E (use surrogate splits) is not XGBoost default. So C and D.

Answer

Use surrogate splits to handle missing values.

Answer

Drop rows with missing values.

Answer

Impute missing values with the mean of the feature.

Question 5

A data scientist is building a deep learning model using Amazon SageMaker. The model is overfitting the training data. Which THREE actions can help reduce overfitting?

Accepted Answer

Add L2 regularization to the loss function.. Overfitting can be reduced by regularization (L2), dropout, data augmentation (increases effective data size), early stopping, and reducing model complexity. Increasing model complexity (more layers) would increase overfitting (Option B). So correct: A, C, D.

Answer

Increase the number of layers in the network.

Answer

Reduce the learning rate.

Question 6

Refer to the exhibit. An IAM policy is attached to a SageMaker notebook instance role. A data scientist is trying to train a model using the SageMaker built-in XGBoost algorithm with training data in 'my-bucket/training-data/' and expects output in 'my-bucket/output/'. The training job fails with an access denied error. What is the most likely missing permission?

Accepted Answer

iam:PassRole on the SageMaker execution role.. The training job uses the SageMaker built-in algorithm, which downloads the training data from S3 and uploads output. The policy allows s3:GetObject on training-data and s3:PutObject on output. However, SageMaker also needs to read the algorithm image from ECR (elasticcontainerregistry). The missing permission is likely ecr:GetDownloadUrlForLayer or ecr:BatchGetImage. Also, SageMaker needs to pass roles. But the error is 'access denied', likely from ECR. Option A (ecr:GetAuthorizationToken) is needed to authenticate, but typically SageMaker uses the role to pull images. Option B (s3:ListBucket) is needed if the training job lists objects. Option C (sagemaker:DescribeTrainingJob) is not needed for execution. Option D (iam:PassRole) is needed for the training job to assume the role. Actually, SageMaker needs iam:PassRole to pass the execution role to the training job. But the error 'access denied' could be due to missing iam:PassRole. However, the policy does not include iam:PassRole. The most likely missing permission is iam:PassRole. Let's check: The policy allows creating training job, but the training job also needs to pass a role to SageMaker. Without iam:PassRole, the API call fails. So D is correct. But also ECR permissions might be needed. However, IAM PassRole is a common missing permission. I'll go with D.

Answer

ecr:GetAuthorizationToken on the ECR repository.

Answer

s3:ListBucket on the S3 bucket.

Answer

sagemaker:DescribeTrainingJob on the training job.

Question 7

Refer to the exhibit. A data scientist ran a SageMaker training job and reviewed the logs. The training completed quickly, but the model performance is very poor. What is the most likely cause?

Accepted Answer

The training dataset is too small for the model complexity.. The training ran for only about 1 minute, which is too short for a typical training. The model likely didn't converge. This indicates that the training job might have been configured with too few epochs or the data was very small, or the algorithm stopped early. The logs show 'Training completed' quickly. The most likely cause is that the training job used a very small number of epochs or early stopping criteria caused premature termination. Option C (model overfitting) would show longer training and high training accuracy. Option D (data leakage) would show good performance. Option A (insufficient training data) could cause poor performance, but the logs show training completed quickly, suggesting the job didn't run long enough. Option B (incorrect learning rate) could cause divergence but would still train for the specified epochs. The quick completion suggests the job was configured with too few epochs or early stopping. But among the options, A (insufficient training data) is plausible. However, the question says 'most likely'. I'll choose B (incorrect learning rate) because if the learning rate is too high, the loss may explode and cause early stopping or NaN, leading to quick termination. But the log doesn't show errors. Actually, the log shows normal completion. So it's likely the model didn't train enough. Option B: If learning rate is too low, training can be slow but still complete epochs. The quick completion suggests the number of epochs was small. But the options don't mention epochs. Option A: insufficient training data would still train for the number of epochs. Option C: overfitting would not cause quick completion. Option D: data leakage would give good performance. So I'm leaning towards B: incorrect learning rate (too high) could cause the loss to become NaN and training to stop, but the log says 'Training completed' not 'Stopped'. It might be that the training completed all epochs because of a small dataset. Actually, the log shows 'Training completed' after 1 minute, so it might have finished all epochs. If the dataset is very small, training could be fast. That would lead to poor performance due to insufficient data. So A is plausible. I'll go with A.

Answer

The model is overfitting to the training data.

Answer

There is data leakage from the test set into the training set.

Answer

The learning rate is too low, causing slow convergence.

Question 8

Refer to the exhibit. A data scientist is using Amazon SageMaker Ground Truth to label a dataset. The output manifest file references S3 objects with metadata. The scientist notices that a training job using the labeled data yields poor accuracy. What is the most likely issue?

Accepted Answer

The training data is in an incorrect format for the algorithm.. The metadata shows 'sagemaker-import-job': 'true', which indicates the object was imported from a SageMaker import job. However, that metadata is not relevant. The content length is 1 GB, which is large. The poor accuracy could be due to many reasons. But the exhibit shows a head-object response, which doesn't directly indicate a problem. However, the question implies that the metadata might be incorrect. Actually, the metadata 'sagemaker-import-job' is set by Ground Truth when importing data. But if the data is not properly labeled, the manifest might be wrong. Option D (data distribution shift) is plausible. Option B (incorrect IAM permissions) would cause access errors. Option C (incorrect data format) could cause issues. Option A (missing labels) is a common Ground Truth issue. But the exhibit doesn't show labels. I think the most likely is that the training data is not representative because the labeling job might have introduced bias. However, I'll choose D (data distribution shift between training and inference). But the question is about the labeled data. Maybe the issue is that the metadata indicates the data was imported but not labeled? Actually, Ground Truth output manifest includes labels. The head-object shows the raw data object, not the manifest. The scientist is looking at the source data. The poor accuracy could be because the data is not properly preprocessed. I'll choose B (incorrect IAM permissions) because if the training job cannot read the manifest, it would fail, but accuracy is poor, not failure. So not that. Option A: missing labels – if the manifest is missing labels, training would fail. Option C: incorrect data format – if the data format is wrong, training might run but produce poor results. That is plausible. I'll go with C.

Answer

The labeled dataset has missing labels for some records.

Answer

The IAM role used for training does not have permissions to read the manifest file.

Answer

The data distribution differs significantly between the training set and the real-world inference data.

Question 9

A data scientist wants to use a linear regression model to predict house prices. After training, the model shows high bias and low variance. Which action would most likely improve the model's performance?

Accepted Answer

Add polynomial features to capture non-linear relationships.. Option C is correct because high bias indicates underfitting, and increasing model complexity (e.g., adding polynomial features or using a more complex algorithm) can reduce bias. Option A is wrong because adding L2 regularization increases bias. Option B is wrong because reducing training data can increase variance but not necessarily reduce bias. Option D is wrong because using a simpler model would increase bias.

Answer

Increase L2 regularization strength.

Answer

Use a simpler model, such as linear regression without interaction terms.

Answer

Reduce the amount of training data.

Question 10

A machine learning team is using SageMaker to train a deep learning model. The training job is failing due to insufficient GPU memory. Which approach should the team take to resolve this issue without changing the model architecture?

Accepted Answer

Use gradient accumulation to reduce the effective batch size per step.. Option B is correct because gradient accumulation simulates larger batch sizes without increasing memory per step. Option A is wrong because increasing batch size increases memory usage. Option C is wrong because reducing learning rate does not affect memory. Option D is wrong because adding more GPUs to a single instance may not help if memory is already exhausted on each GPU; but the key is to reduce per-GPU memory, which gradient accumulation achieves by using a smaller effective batch size per step.

Answer

Increase the batch size.

Answer

Add more GPUs to the training instance.

Answer

Decrease the learning rate.

Question 11

A company uses SageMaker to deploy a model for real-time inference. The model is a large ensemble that requires 8 GB of memory and has high latency. The team wants to reduce latency without increasing cost. Which strategy is most effective?

Accepted Answer

Use SageMaker Neo to compile the model for the target instance.. Option D is correct because SageMaker Neo optimizes trained models for target hardware, reducing latency and memory footprint. Option A is wrong because using a larger instance increases cost. Option B is wrong because batch transform is for offline, not real-time. Option C is wrong because multiple instances increase cost.

Answer

Use a larger instance type with more memory.

Answer

Deploy the model on multiple instances behind a load balancer.

Answer

Switch from real-time inference to batch transform.

Question 12

During training, a binary classification model has an AUC of 0.99 on the training set but only 0.72 on the validation set. Which of the following is the most likely cause?

Accepted Answer

Overfitting.. Option B is correct because a large gap between training and validation performance indicates overfitting. Option A is wrong because underfitting would show poor performance on both. Option C is wrong because data leakage would inflate both metrics. Option D is wrong because class imbalance would affect both sets similarly.

Answer

Class imbalance in the training set.

Answer

Underfitting.

Answer

Data leakage from validation to training.

Question 13

A data scientist is using Amazon SageMaker to train a model using a built-in algorithm. The training job fails with an error indicating that the algorithm expects the data to be in recordIO-protobuf format, but the input is CSV. What is the most efficient way to resolve this?

Accepted Answer

Use a SageMaker processing job to convert the CSV data to recordIO-protobuf format.. Option A is correct because converting CSV to recordIO-protobuf format using a SageMaker processing job is efficient and scalable. Option B is wrong because changing the algorithm may not be desirable. Option C is wrong because recordIO-protobuf is not suitable for CSV. Option D is wrong because direct conversion via boto3 is more manual and less efficient than a processing job.

Answer

Change the inference data to recordIO-protobuf format.

Answer

Use a boto3 script to convert the CSV files locally and upload.

Answer

Switch to a different algorithm that accepts CSV format.

Question 14

A data scientist is training a recurrent neural network (RNN) for time series forecasting. The model's training loss is not decreasing, and the gradients are vanishing. Which technique should the scientist apply to address vanishing gradients?

Accepted Answer

Replace the RNN cells with LSTM or GRU units.. Option C is correct because LSTM and GRU are designed to mitigate vanishing gradients via gating mechanisms. Option A is wrong because gradient clipping addresses exploding gradients, not vanishing. Option B is wrong because increasing learning rate may cause instability. Option D is wrong because batch normalization helps with internal covariate shift but not specifically vanishing gradients.

Answer

Apply gradient clipping.

Answer

Add batch normalization layers.

Answer

Increase the learning rate.

Question 15

A company is building a model to classify customer reviews as positive or negative. The dataset has 10,000 positive and 100 negative reviews. Which metric is most appropriate for evaluating model performance?

Accepted Answer

F1 score.. Option B is correct because F1 score balances precision and recall, important for imbalanced datasets. Option A is wrong because accuracy can be misleading (e.g., 99% by predicting all positive). Option C is wrong because AUC-ROC can be optimistic for imbalanced data. Option D is wrong because mean squared error is for regression.

Answer

Accuracy.

Answer

Mean squared error.

Answer

AUC-ROC.