MLS-C01 Modeling Practice Test 6 — 15 Questions

Question 1

A data scientist is building a regression model to predict house prices. The dataset contains features like number of bedrooms, square footage, and location. After training, the model has high variance. Which technique should the data scientist use to reduce variance without significantly increasing bias?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Apply L2 regularization. L2 regularization (ridge) adds a penalty proportional to the squared magnitude of the model weights to the loss function, which shrinks coefficients and reduces the model's sensitivity to noise in the training data. This directly lowers variance while introducing only a small, controlled amount of bias, which is exactly the trade-off the question asks for. It is the standard, targeted remedy for a high-variance (overfitting) regression model.

Answer

Use bagging

Answer

Increase the number of features

Answer

Use fewer training examples

Question 2

A company wants to build a model to detect fraudulent transactions. The dataset has a highly imbalanced class distribution. Which technique should be used during training to handle class imbalance?

Accepted Answer

Use SageMaker's built-in fraud detection algorithm that applies random under-sampling. SageMaker's built-in fraud detection algorithm (based on the Random Cut Forest or similar anomaly detection approach) applies random under-sampling to balance the class distribution during training, which is a documented technique for handling highly imbalanced fraud datasets. This directly addresses the imbalance by reducing the majority class representation so the model can learn the minority (fraud) patterns.

Answer

Add more features to the dataset

Answer

Reduce the learning rate

Answer

Increase the tree depth in XGBoost

Question 3

A data scientist is working with a dataset containing categorical features with high cardinality. The scientist wants to use a tree-based model. Which encoding method should be used?

Accepted Answer

Label encoding. For tree-based models, label encoding (option C) is typically recommended for high-cardinality categorical features because tree models can handle integer encoding without assuming any order—they split on values, not on ordinal relationships. Ordinal encoding (option A) implies an artificial order that may not exist, potentially misleading the model. One-hot encoding (option D) creates too many dimensions. Target encoding (option B) can cause overfitting, especially with high cardinality.

Answer

Ordinal encoding

Answer

Target encoding

Answer

One-hot encoding

Question 4

Which TWO metrics are appropriate for evaluating a binary classification model when the cost of false negatives is high? (Choose 2)

Accepted Answer

Recall. When the cost of false negatives is high, the primary goal is to minimize false negatives by capturing as many actual positives as possible. Recall (Option A) directly measures this by calculating TP/(TP+FN). Specificity (Option B) measures TN/(TN+FP) and focuses on false positives, not false negatives; it is irrelevant when FN cost is high. Accuracy (C) can be misleading with imbalanced data. F1 score (D) balances precision and recall but does not specifically target FN. Precision (E) focuses on false positives, not false negatives. Therefore, only Recall is appropriate.

Answer

Specificity

Answer

Accuracy

Answer

F1 score

Answer

Precision

Question 5

Which TWO SageMaker features can be used to perform hyperparameter optimization? (Choose 2)

Accepted Answer

SageMaker automatic model tuning. SageMaker automatic model tuning (option D) is the dedicated hyperparameter optimization capability in SageMaker: you define a hyperparameter range or list, choose an objective metric, and it runs a search strategy (Bayesian, Random, Hyperband, or Grid) across multiple training jobs to find the best configuration. SageMaker Experiments (option E) is also correct because it can be used to organize, track, and compare the trials and their metric results produced by hyperparameter tuning jobs, letting you analyze which hyperparameter combinations performed best. The other options do not provide hyperparameter optimization: SageMaker Debugger (A) captures tensors and analyzes training anomalies, SageMaker Pipelines (B) orchestrates ML workflows, and SageMaker Model Monitor (C) detects data and model drift on deployed endpoints.

Answer

SageMaker Debugger

Answer

SageMaker Pipelines

Answer

SageMaker Model Monitor

Question 6

A team is using Amazon SageMaker to train a model and wants to automatically stop training when the model stops improving to save costs. Which SageMaker feature should they use?

Accepted Answer

SageMaker Debugger. SageMaker Debugger provides built-in rules that monitor training jobs and can automatically stop a job when a rule detects an issue, such as a model that is not improving (e.g., the 'StalledTrainingRule' or 'OverfitRule'). This directly addresses the requirement to stop training when the model stops improving to save costs. The other options do not provide automatic early stopping based on model improvement.

Answer

SageMaker Experiments

Answer

SageMaker Managed Spot Training with early stopping

Answer

SageMaker Automatic Model Tuning

Question 7

A data scientist is training a gradient boosting model using SageMaker. The model is overfitting to the training data. Which TWO actions can help reduce overfitting? (Choose 2)

Accepted Answer

Increase the minimum child weight. Option C (Increase the minimum child weight) is correct because in gradient boosting frameworks like XGBoost, the min_child_weight parameter specifies the minimum sum of instance weight (hessian) needed in a child node; raising it makes the tree more conservative by preventing splits that isolate small, noisy subsets of the training data, which directly curbs overfitting. Option D (Reduce the maximum depth of trees) is correct because max_depth controls how deep each tree can grow; shallower trees capture fewer high-order interactions and have lower variance, so reducing it limits model complexity and mitigates overfitting. Option A (Increase the number of boosting rounds) is wrong because adding more boosting iterations lets the ensemble keep fitting residual errors, which increases variance and worsens overfitting. Option B (Increase the learning rate) is wrong because a larger learning rate makes each boosting step more aggressive, causing the model to converge faster to a complex fit and typically amplifying overfitting. Option E (Use a larger training dataset) is not among the marked correct answers; while more data can reduce variance in general, it is not one of the two hyperparameter actions specified here and does not directly address the model-complexity controls that the question targets.

Answer

Increase the number of boosting rounds

Answer

Increase the learning rate

Answer

Use a larger training dataset

Question 8

A company is building a sentiment analysis model using Amazon SageMaker BlazingText. The training data consists of 100,000 product reviews. The data scientist wants to use the Word2Vec algorithm to generate word embeddings. Which configuration is required to use the continuous bag-of-words (CBOW) architecture?

Accepted Answer

Set the mode parameter to 'cbow'.. BlazingText's Word2Vec mode exposes a 'mode' hyperparameter that selects the embedding architecture. Setting mode='cbow' explicitly selects the Continuous Bag-of-Words architecture, where the model predicts a target word from its surrounding context words. This is the documented value for CBOW in SageMaker BlazingText.

Answer

Set the mode parameter to 'supervised'.

Answer

Set the mode parameter to 'batch_skipgram'.

Answer

Set the mode parameter to 'skipgram'.

Question 9

A data scientist is using Amazon SageMaker to train a model. The training job is taking longer than expected. The scientist wants to reduce training time without changing the algorithm or the hardware. Which action is most likely to help?

Accepted Answer

Increase the batch size used during training.. Increasing the batch size allows the training job to process more samples per iteration, which can improve GPU utilization and reduce the number of iterations per epoch, thereby shortening overall training time. This is a hyperparameter change that does not alter the algorithm or hardware. Larger batches also enable more efficient parallelization on the same hardware.

Answer

Add regularization to the loss function.

Answer

Use data augmentation to increase the dataset size.

Answer

Reduce the number of training epochs.

Question 10

A company is using Amazon SageMaker to deploy a model for real-time inference. The model has a latency requirement of less than 100 milliseconds. During testing, the latency is around 150 milliseconds. Which action can most likely reduce the latency to meet the requirement?

Accepted Answer

Use a larger instance type for the endpoint.. Using a larger instance type provides more CPU, memory, and potentially GPU resources, which can accelerate model inference and reduce latency. For a real-time endpoint with a sub-100ms requirement, the current instance may be underpowered, causing the model to take longer to process each request. Upgrading to a more powerful instance directly addresses compute bottlenecks, often yielding the most immediate latency improvement.

Answer

Reduce the batch size for inference.

Answer

Enable data capture for the endpoint.

Answer

Increase the initial variant weight for the production variant.

Question 11

A data scientist is using Amazon SageMaker to train a random forest model for a binary classification task. The dataset has 50 features and 10,000 samples. The model achieves high training accuracy but poor test accuracy. Which TWO actions should the scientist take to improve generalization?

Accepted Answer

Reduce the max_depth of the trees.. The scenario describes overfitting: high training accuracy but poor test accuracy on a 50-feature, 10,000-sample dataset. Option B is correct because reducing max_depth limits how deep each tree can grow, constraining model complexity and reducing variance so the forest generalizes better to unseen data. Option E is correct because increasing min_samples_leaf forces each leaf to contain more samples, smoothing the decision boundaries and preventing the trees from fitting noise in the training set. Option A is incorrect because increasing max_samples (the number of samples drawn per tree) gives each tree more data, which typically reduces variance rather than addressing the overfitting caused by overly complex individual trees. Option C is incorrect because increasing max_features lets each split consider more features, increasing tree correlation and complexity and generally worsening overfitting. Option D is incorrect because adding more trees (n_estimators) mainly stabilizes predictions and reduces variance from averaging, but it does not correct the fundamental overfitting of deep, unconstrained trees.

Answer

Increase the max_samples parameter.

Answer

Increase the max_features parameter.

Answer

Increase the number of trees (n_estimators).

Question 12

A company is using Amazon SageMaker to train an XGBoost model. The training data contains missing values. Which TWO methods can XGBoost handle missing values internally?

Accepted Answer

Learn the best direction to go when a value is missing.. Option C is correct because XGBoost's default behavior for missing values is to learn a default direction at each tree split: during training it evaluates sending missing-value samples to the left or right child and picks the direction that minimizes the loss, so missing values are handled internally without preprocessing. Option D is correct because XGBoost treats missing values as their own distinct group when computing the split gain, effectively assigning them to a separate branch/direction rather than imputing them, which is why it can natively handle sparse and missing data. Option A is not correct because surrogate splits are a technique used by algorithms such as CART/rpart to approximate splits when a predictor is missing, not XGBoost's internal missing-value mechanism. Option B is not correct because dropping rows with missing values is a manual data-preprocessing step, not an internal XGBoost capability. Option E is not correct because mean imputation is an external preprocessing technique and is not how XGBoost handles missing values internally.

Answer

Use surrogate splits to handle missing values.

Answer

Drop rows with missing values.

Answer

Impute missing values with the mean of the feature.

Question 13

Refer to the exhibit. An IAM policy is attached to a SageMaker notebook instance role. A data scientist is trying to train a model using the SageMaker built-in XGBoost algorithm with training data in 'my-bucket/training-data/' and expects output in 'my-bucket/output/'. The training job fails with an access denied error. What is the most likely missing permission?

Accepted Answer

iam:PassRole on the SageMaker execution role.. The training job fails with 'access denied' because the IAM policy attached to the SageMaker notebook instance role does not include the `iam:PassRole` permission. SageMaker requires this permission to pass the execution role specified in the `CreateTrainingJob` API call. Without it, the API call is denied. The other permissions (ECR, S3 list, DescribeTrainingJob) are either not needed at this stage or are already granted via the SageMaker service role, not the notebook role.

Answer

ecr:GetAuthorizationToken on the ECR repository.

Answer

s3:ListBucket on the S3 bucket.

Answer

sagemaker:DescribeTrainingJob on the training job.

Question 14

Refer to the exhibit. A data scientist ran a SageMaker training job and reviewed the logs. The training completed quickly, but the model performance is very poor. What is the most likely cause?

Accepted Answer

The training dataset is too small for the model complexity.. The training job completed very quickly (about 1 minute), which suggests the dataset is small. A small training dataset, especially relative to the model's complexity, leads to poor performance because the model cannot learn generalizable patterns. With insufficient data, the model may overfit the training samples or fail to converge to a good solution, resulting in poor test performance. Other options are less likely: overfitting (A) would typically show high training accuracy but poor validation accuracy, which is not indicated; data leakage (B) would artificially inflate performance; and a low learning rate (C) would cause slow convergence but not necessarily quick completion.

Answer

The model is overfitting to the training data.

Answer

There is data leakage from the test set into the training set.

Answer

The learning rate is too low, causing slow convergence.

Question 15

Refer to the exhibit. A data scientist is using Amazon SageMaker Ground Truth to label a dataset. The output manifest file references S3 objects with metadata. The scientist notices that a training job using the labeled data yields poor accuracy. What is the most likely issue?

Accepted Answer

The training data is in an incorrect format for the algorithm.. The most likely issue is that the training data is in an incorrect format for the algorithm. SageMaker Ground Truth output manifests reference labeled objects in S3, and if the manifest's content type, record format (e.g., JSON Lines vs. augmented manifest), or label schema does not match what the training algorithm expects, the job may run but learn from malformed or misparsed records, producing poor accuracy. Format mismatches are a common and silent failure mode.

Answer

The labeled dataset has missing labels for some records.

Answer

The IAM role used for training does not have permissions to read the manifest file.

Answer

The data distribution differs significantly between the training set and the real-world inference data.