MLS-C01 Modeling Practice Test 17 — 15 Questions

Question 1

A data scientist is using Amazon SageMaker to train a random forest model for a binary classification task. The dataset has 50 features and 10,000 samples. The model achieves high training accuracy but poor test accuracy. Which TWO actions should the scientist take to improve generalization?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Reduce the max_depth of the trees.. The model is overfitting, as indicated by high training accuracy but poor test accuracy. To improve generalization, reduce model complexity. Reducing max_depth (B) limits the depth of each tree, preventing overly specific splits. Increasing min_samples_leaf (E) requires a minimum number of samples per leaf, which smooths the model and reduces variance. These two actions directly combat overfitting in random forests.

Answer

Increase the max_samples parameter.

Answer

Increase the max_features parameter.

Answer

Increase the number of trees (n_estimators).

Question 2

A company is using Amazon SageMaker to train an XGBoost model. The training data contains missing values. Which TWO methods can XGBoost handle missing values internally?

Accepted Answer

Learn the best direction to go when a value is missing.. XGBoost can handle missing values internally using a sparsity-aware algorithm. Option C correctly states that XGBoost learns the best direction to go when a value is missing. Option D correctly states that XGBoost treats missing values as a separate category. Option A (surrogate splits) is not used by XGBoost; Option B (dropping rows) is not an internal method; Option E (imputing with mean) is not an internal method.

Answer

Use surrogate splits to handle missing values.

Answer

Drop rows with missing values.

Answer

Impute missing values with the mean of the feature.

Question 3

A data scientist is building a deep learning model using Amazon SageMaker. The model is overfitting the training data. Which THREE actions can help reduce overfitting?

Accepted Answer

Add L2 regularization to the loss function.. Overfitting can be reduced by regularization techniques such as L2 regularization (Option A) which penalizes large weights, by dropout (Option E) which randomly ignores neurons during training, and by data augmentation (Option B) which increases the effective size of the training dataset by creating modified copies. Increasing model complexity by adding layers (Option C) would worsen overfitting, and reducing the learning rate (Option D) does not directly address overfitting—it affects convergence speed.

Answer

Increase the number of layers in the network.

Answer

Reduce the learning rate.

Question 4

Refer to the exhibit. An IAM policy is attached to a SageMaker notebook instance role. A data scientist is trying to train a model using the SageMaker built-in XGBoost algorithm with training data in 'my-bucket/training-data/' and expects output in 'my-bucket/output/'. The training job fails with an access denied error. What is the most likely missing permission?

Accepted Answer

iam:PassRole on the SageMaker execution role.. The training job fails with 'access denied' because the IAM policy attached to the SageMaker notebook instance role does not include the `iam:PassRole` permission. SageMaker requires this permission to pass the execution role specified in the `CreateTrainingJob` API call. Without it, the API call is denied. The other permissions (ECR, S3 list, DescribeTrainingJob) are either not needed at this stage or are already granted via the SageMaker service role, not the notebook role.

Answer

ecr:GetAuthorizationToken on the ECR repository.

Answer

s3:ListBucket on the S3 bucket.

Answer

sagemaker:DescribeTrainingJob on the training job.

Question 5

Refer to the exhibit. A data scientist ran a SageMaker training job and reviewed the logs. The training completed quickly, but the model performance is very poor. What is the most likely cause?

Accepted Answer

The training dataset is too small for the model complexity.. The training job completed very quickly (about 1 minute), which suggests the dataset is small. A small training dataset, especially relative to the model's complexity, leads to poor performance because the model cannot learn generalizable patterns. With insufficient data, the model may overfit the training samples or fail to converge to a good solution, resulting in poor test performance. Other options are less likely: overfitting (A) would typically show high training accuracy but poor validation accuracy, which is not indicated; data leakage (B) would artificially inflate performance; and a low learning rate (C) would cause slow convergence but not necessarily quick completion.

Answer

The model is overfitting to the training data.

Answer

There is data leakage from the test set into the training set.

Answer

The learning rate is too low, causing slow convergence.

Question 6

Refer to the exhibit. A data scientist is using Amazon SageMaker Ground Truth to label a dataset. The output manifest file references S3 objects with metadata. The scientist notices that a training job using the labeled data yields poor accuracy. What is the most likely issue?

Accepted Answer

The training data is in an incorrect format for the algorithm.. The poor accuracy is most likely due to the training data being in an incorrect format for the algorithm. Amazon SageMaker Ground Truth outputs a manifest file with metadata, but the source S3 objects may be in a format (e.g., raw images, text files) that is not directly compatible with the chosen built-in algorithm or custom model. For example, if the algorithm expects RecordIO-encoded data or a specific CSV structure, but the manifest points to raw JPEG images, the training job will still run (no failure) but produce poor results. Other options: missing labels or IAM issues would typically cause job failures, not just poor accuracy; data distribution shift is possible but less directly indicated by the exhibit.

Answer

The labeled dataset has missing labels for some records.

Answer

The IAM role used for training does not have permissions to read the manifest file.

Answer

The data distribution differs significantly between the training set and the real-world inference data.

Question 7

A data scientist wants to use a linear regression model to predict house prices. After training, the model shows high bias and low variance. Which action would most likely improve the model's performance?

Accepted Answer

Add polynomial features to capture non-linear relationships.. High bias indicates underfitting, meaning the model is too simple to capture underlying patterns. Adding polynomial features increases model complexity, allowing it to better fit the training data and reduce bias. Option B is incorrect because increasing L2 regularization strength penalizes large coefficients, which increases bias and makes underfitting worse. Option C is incorrect because using a simpler model (e.g., linear regression without interaction terms) would further increase bias. Option D is incorrect because reducing training data does not address bias; it can increase variance, but bias remains high.

Answer

Increase L2 regularization strength.

Answer

Use a simpler model, such as linear regression without interaction terms.

Answer

Reduce the amount of training data.

Question 8

A machine learning team is using SageMaker to train a deep learning model. The training job is failing due to insufficient GPU memory. Which approach should the team take to resolve this issue without changing the model architecture?

Accepted Answer

Use gradient accumulation to reduce the effective batch size per step.. Gradient accumulation allows the model to simulate a larger batch size without increasing memory usage per step, as gradients are accumulated over several smaller batches before updating weights. Option A is wrong because increasing the batch size would increase memory consumption per step, exacerbating the issue. Option C (adding more GPUs) is wrong because simply adding more GPUs to the instance does not reduce the memory usage per GPU; each GPU would still process the same batch size and run out of memory. Option D is wrong because decreasing the learning rate does not affect memory usage; it only changes the step size during optimization.

Answer

Increase the batch size.

Answer

Add more GPUs to the training instance.

Answer

Decrease the learning rate.

Question 9

A company uses SageMaker to deploy a model for real-time inference. The model is a large ensemble that requires 8 GB of memory and has high latency. The team wants to reduce latency without increasing cost. Which strategy is most effective?

Accepted Answer

Use SageMaker Neo to compile the model for the target instance.. SageMaker Neo compiles and optimizes the model for the target hardware, reducing latency and memory footprint without increasing cost. Option A is wrong because a larger instance would increase cost. Option B is wrong because deploying on multiple instances increases cost and complexity without necessarily reducing latency. Option D is wrong because batch transform is for offline inference, not real-time, and does not address the low latency requirement.

Answer

Use a larger instance type with more memory.

Answer

Deploy the model on multiple instances behind a load balancer.

Answer

Switch from real-time inference to batch transform.

Question 10

During training, a binary classification model has an AUC of 0.99 on the training set but only 0.72 on the validation set. Which of the following is the most likely cause?

Accepted Answer

Overfitting.. A large gap between high training AUC (0.99) and lower validation AUC (0.72) indicates overfitting. Option A is wrong: class imbalance would affect both sets similarly or the model might ignore the minority class, but would not typically produce such a large gap. Option B is wrong: underfitting would show poor performance on both sets (e.g., AUC around 0.5-0.6). Option D is wrong: data leakage would inflate both training and validation metrics, not create a gap.

Answer

Class imbalance in the training set.

Answer

Underfitting.

Answer

Data leakage from validation to training.

Question 11

A data scientist is using Amazon SageMaker to train a random forest model for a binary classification task. The dataset has 50 features and 10,000 samples. The model achieves high training accuracy but poor test accuracy. Which TWO actions should the scientist take to improve generalization?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use a SageMaker processing job to convert the CSV data to recordIO-protobuf format.. Using a SageMaker processing job to convert CSV data to recordIO-protobuf format is an efficient, scalable, and fully managed solution within the SageMaker ecosystem. Option A is incorrect because changing the inference data to recordIO-protobuf does not address the training data format issue. Option B is incorrect because using a boto3 script to convert locally is less efficient and not scalable compared to a managed processing job. Option D is incorrect because switching algorithms may not be desirable and does not solve the underlying data format requirement.

Answer

Change the inference data to recordIO-protobuf format.

Answer

Use a boto3 script to convert the CSV files locally and upload.

Answer

Switch to a different algorithm that accepts CSV format.

Question 12

A data scientist is training a recurrent neural network (RNN) for time series forecasting. The model's training loss is not decreasing, and the gradients are vanishing. Which technique should the scientist apply to address vanishing gradients?

Accepted Answer

Replace the RNN cells with LSTM or GRU units.. LSTM and GRU units incorporate gating mechanisms (input, forget, output gates) that regulate the flow of gradients, effectively mitigating the vanishing gradient problem. Option A is wrong because gradient clipping is used to prevent exploding gradients, not vanishing gradients. Option C is wrong because batch normalization helps stabilize training by reducing internal covariate shift, but it does not specifically address vanishing gradients. Option D is wrong because increasing the learning rate may cause the training to diverge or become unstable, and it does not solve the vanishing gradient issue.

Answer

Apply gradient clipping.

Answer

Add batch normalization layers.

Answer

Increase the learning rate.

Question 13

A company is building a model to classify customer reviews as positive or negative. The dataset has 10,000 positive and 100 negative reviews. Which metric is most appropriate for evaluating model performance?

Accepted Answer

F1 score.. (F1 score) is most appropriate because it balances precision and recall, making it suitable for imbalanced datasets where accuracy can be misleading (e.g., a model predicting all positive would achieve 99% accuracy). Option B (Accuracy) is misleading in imbalanced scenarios. Option C (Mean squared error) is for regression, not classification. Option D (AUC-ROC) can be used but may provide an overly optimistic assessment in highly imbalanced data.

Answer

Accuracy.

Answer

Mean squared error.

Answer

AUC-ROC.

Question 14

A team is using SageMaker to train a model with hyperparameter tuning. The training jobs are taking too long. The team wants to reduce time without sacrificing model quality. Which approach should they take?

Accepted Answer

Enable early stopping in the hyperparameter tuning job.. Enabling early stopping terminates poorly performing training jobs early, saving time without sacrificing model quality. Option A is incorrect: random search may be faster but does not guarantee quality as it is less efficient than Bayesian optimization. Option C is incorrect: increasing the maximum number of training jobs would increase time. Option D is incorrect: reducing the maximum runtime per training job may prevent convergence, harming model quality.

Answer

Use random search instead of Bayesian optimization.

Answer

Increase the maximum number of training jobs.

Answer

Reduce the maximum runtime per training job.

Question 15

A data scientist is using Amazon SageMaker to train a random forest model for a binary classification task. The dataset has 50 features and 10,000 samples. The model achieves high training accuracy but poor test accuracy. Which TWO actions should the scientist take to improve generalization?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use a real-time endpoint with an instance that has sufficient memory.. For a large model requiring 16GB memory and minimal inference latency, a real-time endpoint with a suitably sized instance (e.g., ml.p3.2xlarge or ml.g4dn.xlarge) provides dedicated resources and low latency. Option B (asynchronous inference) adds queuing latency and is for non-real-time. Option C (Serverless Inference) has memory limits (up to 6 GB) and may have cold starts, not suitable for a 16GB model. Option D (batch transform) is for offline inference on batches, not real-time.

Answer

Use an asynchronous inference endpoint.

Answer

Use SageMaker Serverless Inference.

Answer

Use a batch transform job.