MLS-C01 Modeling Practice Test 14 — 15 Questions

Question 1

An IAM policy attached to a SageMaker execution role is shown. A training job executed with this role fails with an error that the role cannot access the S3 bucket. The training job uses input data from s3://my-bucket/train/data.csv and output to s3://my-bucket/output/. What is the most likely cause?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The training job does not have s3:PutObject permission for the output location. The error message indicates the role cannot access the S3 bucket, which typically occurs when the role lacks write permissions to the output location. The training job needs s3:PutObject permission to write the output artifacts (model, logs, etc.) to s3://my-bucket/output/. Without this permission, SageMaker fails to save the training results, resulting in an access error.

Answer

The training job does not have s3:GetObject permission for the input data

Answer

The training data is encrypted with SSE-KMS and the role lacks KMS permissions

Answer

The S3 bucket is in a different region than the training job

Question 2

A data scientist needs to evaluate a binary classification model. The dataset is balanced. Which metric is most appropriate to compare model performance?

Accepted Answer

Accuracy. For a balanced binary classification dataset, accuracy is the most appropriate metric because it directly measures the proportion of correct predictions (true positives and true negatives) out of all predictions. Since the class distribution is equal, accuracy is not misleadingly high due to class imbalance, making it a reliable and straightforward measure of overall model performance.

Answer

Recall

Answer

F1 score

Answer

Precision

Question 3

A company is deploying a machine learning model for fraud detection. The model outputs a probability score. The cost of false negatives is very high. Which TWO metrics should the company focus on optimizing?

Accepted Answer

F1 score. Recall (Option E) is correct because it measures the proportion of actual fraud cases that the model successfully identifies (TP / (TP + FN)); when false negatives are very costly, maximizing recall directly minimizes the number of missed frauds. The F1 score (Option C) is also correct because it is the harmonic mean of precision and recall (2 * (Precision * Recall) / (Precision + Recall)), so it rewards models that maintain high recall while still keeping precision reasonable, giving a balanced metric that is sensitive to the costly false negatives. Precision (Option A) focuses on the proportion of predicted frauds that are truly fraud (TP / (TP + FP)) and ignores false negatives, so it does not directly address the high cost of misses. False positive rate (Option B) measures FP / (FP + TN) and is concerned with false alarms rather than missed frauds. AUC-ROC (Option D) summarizes performance across all thresholds but can be misleadingly high on imbalanced fraud data and does not specifically prioritize the low false-negative requirement.

Answer

Precision

Answer

False positive rate (FPR)

Answer

Area under the ROC curve (AUC-ROC)

Question 4

A data scientist is evaluating a classification model. The confusion matrix shows that the model has 50 true positives, 100 true negatives, 20 false positives, and 30 false negatives. Which TWO metrics can be calculated from this confusion matrix? (Choose two.)

Accepted Answer

Recall. Recall (Option C) is correct because it is calculated as TP / (TP + FN) = 50 / (50 + 30) = 0.625, and all required values come directly from the confusion matrix. Precision (Option E) is correct because it is calculated as TP / (TP + FP) = 50 / (50 + 20) = 0.714, again using only confusion-matrix counts. R-squared (Option A) is not applicable because it measures goodness of fit for regression models, not classification. Root mean squared error (Option D) is also a regression metric based on continuous prediction errors, so it cannot be derived from a confusion matrix. F1 score (Option B) is not marked correct here even though it is technically computable from precision and recall; per the question's answer key, only Recall and Precision are the intended selections.

Answer

R-squared

Answer

F1 score

Answer

Root mean squared error

Question 5

A financial services company is developing a fraud detection model using a highly imbalanced dataset where fraudulent transactions are only 0.1% of the data. The data scientist has trained a gradient boosting model that achieves 99.9% accuracy but only detects 20% of actual fraud cases. The business requirement is to detect at least 80% of fraud while minimizing false positives. The data scientist has access to SageMaker and can use any built-in algorithm or custom script. Which approach should the data scientist take to meet the business requirement?

Accepted Answer

Use Amazon SageMaker Random Cut Forest (RCF) algorithm for anomaly detection.. Amazon SageMaker Random Cut Forest (RCF) is an unsupervised anomaly detection algorithm that is well-suited for highly imbalanced datasets like this one (0.1% fraud). Unlike supervised methods that struggle with extreme class imbalance, RCF isolates anomalies by measuring how many random cuts are needed to separate a point from the rest of the data, making it effective at detecting rare fraud cases without requiring balanced training data. This approach can meet the 80% fraud detection requirement while minimizing false positives by tuning the anomaly score threshold.

Answer

Keep the model but adjust the classification threshold to increase recall.

Answer

Use random under-sampling of the majority class to balance the dataset and retrain the model.

Answer

Use random oversampling of the minority class to balance the dataset and retrain the model.

Question 6

A data scientist is using Amazon SageMaker to train a linear regression model. The training data has 10 features and 100,000 observations. The model's training loss is decreasing, but the validation loss starts increasing after a few epochs. Which step should the data scientist take first to address this issue?

Accepted Answer

Reduce the learning rate. The increasing validation loss while training loss decreases is a classic sign of overfitting. Reducing the learning rate (Option B) is the first step to stabilize training by allowing the optimizer to take smaller, more controlled steps, which can help the model converge to a better local minimum and reduce validation loss. In SageMaker, this is typically adjusted via the `learning_rate` hyperparameter in the estimator.

Answer

Add more features to the model

Answer

Increase the batch size

Answer

Increase the number of epochs

Question 7

A data scientist is training a binary classifier on a dataset with 1 million rows and 500 features. The model uses XGBoost and achieves an AUC of 0.95 on the training set but only 0.72 on the test set. The scientist suspects overfitting. Which combination of hyperparameter adjustments is most likely to improve generalization?

Accepted Answer

Decrease 'max_depth' and increase 'min_child_weight'. Decreasing 'max_depth' reduces the complexity of individual trees, preventing the model from learning overly specific patterns in the training data. Increasing 'min_child_weight' forces the algorithm to require a higher sum of instance weights (hessian) before further partitioning, which acts as a regularization mechanism that discourages splits on noisy or sparse data. Together, these adjustments directly combat overfitting in XGBoost by limiting tree depth and requiring more evidence for splits, which improves generalization from the training AUC of 0.95 to a higher test AUC.

Answer

Increase 'max_depth' and decrease 'learning_rate'

Answer

Increase 'subsample' and decrease 'colsample_bytree'

Answer

Decrease 'gamma' and increase 'learning_rate'

Question 8

A data scientist is building a text classification model using a bag-of-words approach with logistic regression. The dataset has 10,000 documents and 50,000 unique tokens. The model overfits. Which TWO techniques can help reduce overfitting?

Accepted Answer

Reduce the vocabulary size by removing rare and very frequent terms. Option D is correct because reducing the vocabulary size by removing rare and very frequent terms (e.g., using min_df/max_df thresholds or stop-word removal) lowers the feature dimensionality from 50,000 tokens, which directly reduces model variance and the risk of fitting noise in a bag-of-words representation. Option E is correct because applying L2 regularization (ridge, controlled by the C or alpha hyperparameter in scikit-learn's LogisticRegression) penalizes large coefficients, shrinking the weights toward zero and constraining the model's capacity so it generalizes better on the 10,000 documents. The unmarked options do not belong: A increases n-gram features and thus dimensionality, worsening overfitting; B one-hot encoding is essentially equivalent to a binary bag-of-words and does not reduce feature count or variance; and C a more complex neural network increases model capacity, which typically aggravates overfitting rather than reducing it.

Answer

Increase the number of n-grams features

Answer

Use one-hot encoding instead of bag-of-words

Answer

Use a more complex model such as a neural network

Question 9

A company is training a deep learning model for object detection using Amazon SageMaker. The training job is taking too long. Which TWO actions can reduce training time?

Accepted Answer

Use distributed training with multiple GPUs. Option A is correct because distributed training with multiple GPUs parallelizes the forward and backward passes across devices (e.g., via SageMaker distributed data parallel or Horovod), dramatically increasing throughput and reducing wall-clock training time for deep learning workloads. Option D is correct because a smaller initial batch size with gradual warm-up lets the model take more optimization steps early and stabilizes convergence, which often shortens the total time to reach target accuracy compared to a fixed large batch that may need more epochs or tuning. Option B is not necessarily effective because object detection training is typically GPU-bound, so adding vCPUs on a CPU-heavy instance won't speed up the GPU compute. Option C (managed spot training) reduces cost, not training time, and can even add interruption/checkpoint overhead. Option E (more epochs) increases training time rather than reducing it.

Answer

Use a larger instance type with more vCPUs

Answer

Use SageMaker managed spot training

Answer

Increase the number of epochs

Question 10

A data scientist is using Amazon SageMaker to train a linear regression model. The training data contains missing values. Which TWO techniques are appropriate for handling missing values in the dataset?

Accepted Answer

Remove rows with missing values if the proportion is small. Option C is correct because listwise deletion (dropping rows with missing values) is a standard and valid preprocessing technique when the missing-data proportion is small, since it removes only a negligible amount of information and avoids introducing bias from imputation. Option D is correct because mean imputation replaces missing entries in a numeric column with that column's average, which preserves the sample size and is a widely accepted approach for continuous features such as those used in linear regression. Option A is not appropriate here because the scenario specifies a linear regression model, and decision trees' native missing-value handling does not apply to linear regression. Option B is not appropriate because setting missing values to zero arbitrarily distorts the distribution and biases the regression coefficients unless zero is a meaningful value. Option E is not appropriate because creating a separate category for missing values applies to categorical features, whereas linear regression requires numeric inputs.

Answer

Use a decision tree model that can handle missing values internally

Answer

Set missing values to zero

Answer

Create a separate category for missing values

Question 11

A data scientist is working on a binary classification problem to predict loan default. The dataset has 200,000 samples and 50 features. The target variable is imbalanced: 5% default, 95% non-default. The scientist trains a logistic regression model and achieves 95% accuracy, but the recall for the default class is only 20%. The business requires that at least 70% of actual defaults be identified (recall >= 0.7). Which approach should the scientist take to improve recall without significantly sacrificing precision?

Accepted Answer

Use oversampling techniques like SMOTE to create synthetic samples of the minority class. SMOTE (Synthetic Minority Oversampling Technique) creates synthetic samples for the minority class by interpolating between existing minority instances, which increases the representation of the default class in the training data. This directly addresses the low recall (20%) by providing the logistic regression model with more balanced class distributions, enabling it to learn decision boundaries that capture more true positives without discarding majority class information. Unlike simple oversampling, SMOTE reduces overfitting risk by generating novel samples rather than duplicating existing ones, which helps maintain precision while improving recall.

Answer

Use random undersampling of the majority class to balance the dataset

Answer

Change the decision threshold to 0.3

Answer

Increase the regularization strength (C) in logistic regression

Question 12

A machine learning team is deploying a model for real-time fraud detection. The model must make predictions with less than 100ms latency. The team uses SageMaker and the model is a large ensemble of decision trees. Which SageMaker hosting option is MOST suitable?

Accepted Answer

SageMaker Serverless Inference. SageMaker Serverless Inference is the most suitable option because it automatically scales to handle variable traffic and does not require managing underlying infrastructure. Although it may incur cold starts, for a constantly invoked fraud detection model the endpoint remains warm, achieving sub-100ms latency. The large ensemble of decision trees can be deployed as a single model on a Serverless endpoint, which is optimized for real-time inference with low latency and automatic scaling.

Answer

SageMaker Multi-model endpoint

Answer

SageMaker Elastic Inference

Answer

SageMaker Batch Transform

Question 13

A company uses SageMaker to train a model that processes sensitive customer data. Due to compliance, the training data must be encrypted at rest and in transit, and the model artifacts must be stored in a secured S3 bucket with encryption. Which combination of actions is REQUIRED?

Accepted Answer

Enable AWS KMS encryption for the SageMaker notebook and training job, and use an S3 bucket with default encryption using AWS KMS. It ensures end-to-end encryption: AWS KMS encryption for the SageMaker notebook and training job encrypts data in transit and at rest within the SageMaker environment, while an S3 bucket with default encryption using AWS KMS encrypts the training data and model artifacts at rest. This combination meets compliance requirements for encryption at rest and in transit, as KMS provides envelope encryption with customer-managed keys, and SageMaker automatically uses TLS for data in transit when KMS is enabled.

Answer

Store data in an S3 bucket with AWS CloudHSM integration

Answer

Use an S3 bucket with SSE-S3 and enable SageMaker Internet-facing mode

Answer

Use an S3 bucket with default encryption (SSE-S3) and enable SSL for all connections

Question 14

A data scientist is training a model using SageMaker's built-in XGBoost algorithm. The dataset has 500 features and 1 million rows. The training job is taking too long. The scientist wants to reduce training time without sacrificing accuracy. Which action is LIKELY to be most effective?

Accepted Answer

Apply Principal Component Analysis (PCA) to reduce the number of features. PCA reduces the dimensionality of the dataset from 500 features to a smaller set of principal components, which directly decreases the computational complexity of training the XGBoost model. With fewer features, each tree split requires fewer operations, and the overall training time drops significantly. Since PCA retains the variance in the data, it can preserve model accuracy while speeding up training.

Answer

Use a smaller instance type to reduce time

Answer

Reduce the number of trees in XGBoost

Answer

Use a larger instance type with more vCPUs

Question 15

A data scientist is building a binary classifier to predict customer churn. The dataset is highly imbalanced (5% churn). Which TWO techniques can help improve the model's ability to detect churn?

Accepted Answer

Use Synthetic Minority Over-sampling Technique (SMOTE). Option B (SMOTE) is correct because it generates synthetic examples of the minority (churn) class by interpolating between existing minority instances and their nearest neighbors, which balances the class distribution and helps the classifier learn the minority decision boundary. Option C (class weights in the loss function) is correct because assigning a higher penalty to misclassified minority-class samples directly counteracts the imbalance during training, making the model more sensitive to churn cases. Option A is not marked correct: downsampling the majority class discards potentially useful data and can degrade overall performance, and it is not one of the selected answers here. Option D is wrong because accuracy is misleading on imbalanced data—predicting the majority class alone yields 95% accuracy while detecting no churn—so metrics like precision, recall, F1, or AUC-ROC should be used instead. Option E is wrong because simply adding layers increases model complexity and risks overfitting without addressing the class imbalance itself.

Answer

Downsample the majority class to balance the dataset

Answer

Use accuracy as the evaluation metric

Answer

Increase the model complexity by adding more layers