MLS-C01 Modeling Practice Test 2 — 15 Questions

Question 1

A data scientist is trying to run a SageMaker training job that writes output to an S3 bucket 'my-bucket'. The IAM policy is shown. The training job fails with an AccessDenied error when trying to write to S3. What is the reason?

Accepted Answer

The S3 bucket is encrypted with AWS KMS and the policy does not include kms:GenerateDataKey. The correct answer is A because the training job fails with an AccessDenied error when writing to an S3 bucket that is encrypted with AWS KMS. The IAM policy shown must include the `kms:GenerateDataKey` permission to allow the SageMaker training job to generate a data key for encrypting the output objects. Without this KMS permission, the S3 PutObject operation is denied even if the policy allows `s3:PutObject`, as KMS encryption requires explicit authorization to use the customer master key (CMK).

Answer

The policy does not allow s3:ListBucket

Answer

The policy does not allow s3:PutObject

Answer

The policy does not allow s3:PutObjectAcl

Question 2

A data scientist is training a binary classifier to predict customer churn. The dataset has 10,000 samples, with 500 churners (positive class). The scientist trains a logistic regression model and obtains an F1-score of 0.6. To improve the F1-score, which approach is MOST likely to be effective?

Accepted Answer

Apply SMOTE to oversample the minority class. The dataset is highly imbalanced (500 churners out of 10,000 samples, a 5% positive rate). Logistic regression trained on such imbalance tends to bias toward the majority class, resulting in low recall for the minority class and a poor F1-score. SMOTE (Synthetic Minority Oversampling Technique) generates synthetic samples for the minority class by interpolating between existing minority instances, which balances the class distribution and allows the model to learn a better decision boundary, directly improving recall and F1-score.

Answer

Increase the regularization strength (C)

Answer

Apply PCA to reduce feature dimensionality

Answer

Use the original dataset without any modification

Question 3

A company is deploying a real-time fraud detection system using a gradient boosting model on AWS SageMaker. The model uses 200 features and is trained on 50 GB of data. The inference latency requirement is under 10 ms per request. During load testing, the endpoint shows average latency of 15 ms. Which change is MOST likely to reduce latency below 10 ms?

Accepted Answer

Reduce the number of features to the top 50 based on feature importance. Reducing the number of features from 200 to the top 50 directly decreases the amount of data each inference request must process, which lowers both feature engineering overhead and model evaluation time. For gradient boosting models on SageMaker, fewer features mean fewer decision tree splits to traverse per prediction, which can significantly reduce latency without requiring hardware changes. This is the most direct and cost-effective way to meet the 10 ms requirement.

Answer

Switch to a GPU-based instance type

Answer

Increase the number of trees in the model

Answer

Use a larger batch size for inference

Question 4

A machine learning team is training a deep learning model on Amazon SageMaker and notices that the training loss is decreasing but the validation loss is increasing. What is the most likely cause?

Accepted Answer

Overfitting the training data. When training loss continues to decrease while validation loss increases, the model is memorizing the training data rather than learning generalizable patterns. This is the classic symptom of overfitting, where the model's capacity exceeds what is needed for the underlying data distribution, causing it to fit noise in the training set. In Amazon SageMaker, this can be observed by monitoring the validation loss metric during training jobs.

Answer

Vanishing gradients

Answer

Learning rate is too high

Answer

Underfitting the training data

Question 5

A company is building a recommendation system for an e-commerce platform. They have user-item interaction data (clicks, purchases) and want to use matrix factorization. They plan to use Amazon SageMaker to train the model. Which dataset format is MOST appropriate for the built-in Factorization Machines algorithm?

Accepted Answer

RecordIO-protobuf with user_id, item_id, and label fields. The built-in Factorization Machines algorithm in Amazon SageMaker requires the RecordIO-protobuf format for optimal performance, as it allows efficient binary serialization and direct integration with SageMaker's distributed training infrastructure. This format supports sparse data representation, which is critical for high-dimensional user-item interaction data, and enables faster I/O and reduced memory overhead compared to text-based formats.

Answer

Libsvm format with user_id and item_id as features

Answer

CSV file with user_id, item_id, and label columns

Answer

JSON lines file with user_id, item_id, and label fields

Question 6

A data scientist is tuning a gradient boosting model using Amazon SageMaker's Automatic Model Tuning (hyperparameter optimization). The objective metric is validation:auc. After 50 training jobs, the best model still has a validation AUC of only 0.65. The scientist suspects overfitting because the training AUC is 0.99. Which hyperparameter configuration is MOST likely to reduce overfitting?

Accepted Answer

Increase lambda from 1 to 10. Increasing lambda (L2 regularization) from 1 to 10 adds a stronger penalty on the magnitude of leaf weights in the gradient boosting model. This directly reduces overfitting by discouraging the model from fitting noise in the training data, which is consistent with the observed gap between training AUC (0.99) and validation AUC (0.65). In XGBoost, lambda controls the L2 regularization term on weights, and a higher value forces the model to be simpler and more generalizable.

Answer

Increase num_round from 100 to 500

Answer

Increase max_depth from 6 to 12

Answer

Increase subsample from 0.5 to 1.0

Question 7

A data scientist is training a neural network for image classification. The training loss is not decreasing significantly, and the validation loss is high. Which TWO actions should the scientist take to address potential vanishing gradients?

Accepted Answer

Use ReLU activation functions in hidden layers. ReLU activation functions help mitigate vanishing gradients because they output a constant gradient of 1 for positive inputs, preventing the gradient from shrinking as it propagates backward through many layers. This avoids the exponential decay of gradients that occurs with saturating activations like sigmoid or tanh, enabling effective training of deep networks.

Answer

Increase the learning rate

Answer

Switch activation functions from ReLU to sigmoid

Answer

Remove dropout layers

Question 8

A company is using Amazon SageMaker to train a large language model. The training job is taking too long. The data scientist wants to reduce training time without sacrificing model accuracy. Which THREE strategies are MOST appropriate?

Accepted Answer

Use mixed precision training (float16). Mixed precision training (float16) reduces memory usage and accelerates computation by using half-precision floating-point numbers for most operations, while maintaining a single-precision copy of critical parameters to preserve accuracy. This directly reduces training time on compatible GPUs (e.g., NVIDIA V100, A100) without sacrificing model quality, as the loss scaling technique prevents underflow in gradients.

Answer

Switch from GPU instance to CPU instance

Answer

Increase the maximum sequence length

Question 9

A data scientist is training a binary classification model on a dataset with a severe class imbalance (95% negative, 5% positive). The model achieves 95% accuracy but only correctly identifies 10% of the positive class. Which metric should the data scientist use to evaluate model performance?

Accepted Answer

F1 score. The F1 score is the harmonic mean of precision and recall, making it robust to class imbalance. With 95% accuracy but only 10% recall on the positive class, the model is essentially a trivial classifier that predicts the majority class. F1 score captures both false positives and false negatives, providing a balanced view of performance on the minority class.

Answer

Log loss

Answer

Accuracy

Answer

Area under the ROC curve (AUC)

Question 10

A company is building a recommendation system for an e-commerce platform. They have user-item interaction data and want to use matrix factorization. However, the dataset is sparse (99% missing interactions). Which approach should the data scientist take to train the model effectively?

Accepted Answer

Use alternating least squares (ALS) with implicit feedback and assign lower confidence to unobserved interactions. Option B is correct because Alternating Least Squares (ALS) with implicit feedback is specifically designed to handle sparse implicit feedback datasets by assigning lower confidence to unobserved interactions (e.g., confidence = 1 + alpha * r_ui, where r_ui is 0 for unobserved). This avoids the pitfalls of treating missing values as zeros (which distorts the factorization) and scales well to 99% sparsity by leveraging weighted regularization.

Answer

Impute missing values with zeros and use singular value decomposition (SVD)

Answer

Remove all users and items with fewer than 10 interactions to reduce sparsity

Answer

Use item-based collaborative filtering with cosine similarity

Question 11

A financial services company is building a model to detect fraudulent credit card transactions. The dataset contains 1 million transactions, with only 0.1% labeled as fraud. The data scientist trains a logistic regression model on the raw dataset and obtains the following results on a held-out test set: accuracy = 99.8%, precision = 50%, recall = 60%, F1 = 0.545. The business requirement is to maximize recall while keeping precision above 80%. Which course of action should the data scientist take to improve the model?

Accepted Answer

Train the model with class weights inversely proportional to class frequencies. The correct answer is C because assigning class weights inversely proportional to class frequencies penalizes misclassifications of the minority class (fraud) more heavily during training. This directly addresses the severe class imbalance (0.1% fraud) by forcing the logistic regression model to learn decision boundaries that improve recall, while the weight ratio can be tuned to maintain precision above 80%. Unlike naive resampling, this approach preserves the original data distribution and avoids information loss.

Answer

Use random undersampling of the majority class to balance the dataset

Answer

Collect more historical transaction data and retrain the model

Answer

Apply L2 regularization with a higher penalty to reduce overfitting

Question 12

A research team is developing a deep learning model to classify medical images into 10 disease categories. They have a dataset of 50,000 labeled images, but the class distribution is highly imbalanced: the most common class has 20,000 images, while the rarest class has only 200 images. To address this, they apply data augmentation (random rotations, flips, and brightness adjustments) to the minority classes until each class has 20,000 images. They then train a convolutional neural network (CNN) from scratch using cross-entropy loss. The model achieves 95% overall accuracy but only 30% recall on the rarest class. Which change is MOST likely to improve recall on the rarest class without significantly reducing overall accuracy?

Accepted Answer

Replace cross-entropy loss with focal loss. Focal loss is specifically designed to address class imbalance by down-weighting the loss contribution from well-classified examples (majority classes) and focusing training on hard, misclassified examples (minority classes). This directly improves recall on the rarest class, while cross-entropy loss treats all classes equally, causing the model to be biased toward the majority classes.

Answer

Increase dropout rate from 0.2 to 0.5 to reduce overfitting

Answer

Switch from Adam optimizer to SGD with momentum

Answer

Reduce the batch size from 64 to 16 to increase stochasticity

Question 13

A data scientist is training a linear regression model to predict house prices. The dataset includes features such as square footage, number of bedrooms, and location. After training, the model achieves an R² of 0.85 on the training set but only 0.60 on the test set. Which of the following is the MOST likely cause of this discrepancy?

Accepted Answer

The model is overfitting the training data. A high R² on the training set (0.85) paired with a significantly lower R² on the test set (0.60) is a classic symptom of overfitting. The model has learned noise and specific patterns in the training data that do not generalize to unseen data, causing poor test performance. Regularization techniques like Lasso or Ridge, or reducing model complexity, would typically address this issue.

Answer

There is multicollinearity among the features

Answer

The model is underfitting the training data

Answer

There is data leakage between the training and test sets

Question 14

A machine learning team is building a multi-class image classifier using a pre-trained ResNet-50 model in Amazon SageMaker. The dataset has 10 classes but is highly imbalanced, with one class representing 80% of the samples. The team wants to improve model performance on the minority classes. Which TWO of the following approaches are most likely to help? (Select TWO.)

Accepted Answer

Oversample the minority classes in the training data.. Oversampling the minority classes (Option A) directly addresses class imbalance by replicating samples from underrepresented classes, giving the model more exposure to them during training. This is a standard data-level technique that helps the ResNet-50 model learn discriminative features for minority classes without altering the loss function or model architecture.

Answer

Reduce the batch size to increase the frequency of weight updates.

Answer

Increase the number of layers in the model.

Answer

Switch to a focal loss function.

Question 15

A financial services company is building a fraud detection model using a large dataset of credit card transactions. The dataset contains 10 million rows with 50 features, including transaction amount, merchant category, time of day, and customer historical features. The label is binary: fraudulent (1% of data) or legitimate. The company wants to deploy a real-time inference endpoint using Amazon SageMaker that can score transactions with sub-100ms latency. The current model is a gradient boosting model (XGBoost) trained on a sample of 1 million rows due to memory constraints. The model achieves 0.95 AUC on a held-out test set but the fraud recall (sensitivity) is only 0.4, which is unacceptable because the cost of missing a fraud is high. The data science team has access to a larger compute instance (ml.m5.24xlarge) for training. Which course of action is most likely to improve fraud recall while maintaining latency requirements?

Accepted Answer

Train the XGBoost model on the full 10 million rows using an ml.p3.2xlarge instance with GPU support, and apply SMOTE oversampling to the minority class before training.. Option A is correct because training on the full 10 million rows with a GPU-accelerated instance (ml.p3.2xlarge) allows the XGBoost model to learn from the complete data distribution, addressing the bias introduced by the 1 million row sample. Applying SMOTE oversampling to the minority class (fraud) directly tackles the class imbalance (1% fraud), which is the root cause of the low recall (0.4). SMOTE generates synthetic fraudulent examples, improving the model's ability to detect fraud without significantly increasing inference latency, as the model architecture and deployment remain unchanged.

Answer

Engineer additional features from transaction time and merchant category, then retrain the XGBoost model on the same 1 million row sample.

Answer

Downsample the majority class to 1% of the original size to create a balanced dataset of 200,000 rows, then retrain the XGBoost model on this balanced sample.

Answer

Replace XGBoost with a logistic regression model trained on the full dataset, as linear models are faster to train and may generalize better on large data.

MLS-C01 Modeling Practice Test 2 — 15 Questions

Exhibit

MLS-C01 Modeling Practice Test 2 — 15 Questions

Exhibit