MLS-C01 Practice Test 15 — 15 Questions

Question 1

A company is building a binary classifier to predict equipment failure. The dataset has 99% negative (no failure) and 1% positive (failure) examples. The data scientist uses a random forest model with default settings. The model achieves 99% accuracy on the test set but fails to identify any actual failures. Which metric should the data scientist use to evaluate the model?

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual positive cases correctly identified. With 99% negative examples, a model can achieve 99% accuracy by simply predicting 'no failure' for all instances, but this yields 0% recall for the failure class. Since the goal is to detect rare failures, recall is the appropriate metric to evaluate the model's ability to find positive cases.

Answer

RMSE

Answer

R-squared

Answer

Precision

Question 2

A company is building a recommendation system for an e-commerce platform. The data includes user-item interactions and features such as user demographics and item categories. Which algorithm would be most appropriate for generating personalized recommendations?

Accepted Answer

Factorization Machines. Factorization Machines (FM) are specifically designed for recommendation tasks with sparse, high-dimensional data like user-item interactions. They model pairwise feature interactions (e.g., user demographics × item categories) using factorized parameters, enabling personalized recommendations even when many user-item pairs are unobserved. This makes FM far more effective than tree-based or clustering methods for collaborative filtering and feature-rich recommendation scenarios.

Answer

XGBoost

Answer

k-means clustering

Answer

Principal Component Analysis (PCA)

Question 3

Refer to the exhibit. A data scientist is trying to run a SageMaker training job using a script that reads data from the S3 bucket 'my-bucket' and writes the model artifact to the same bucket. The training job fails with an access denied error. What is the likely cause?

Accepted Answer

The IAM role does not have permission to write to the S3 bucket for the model artifact. The training job fails with an access denied error because the IAM role used by SageMaker lacks the s3:PutObject permission (or equivalent write access) for the S3 bucket 'my-bucket'. While the script reads data from the bucket, writing the model artifact requires explicit write permissions on the same bucket. Without this, SageMaker cannot upload the model artifact, causing the job to fail.

Answer

The IAM role does not have sagemaker:CreateModel permission

Answer

The IAM role does not have s3:ListBucket permission

Answer

The IAM role does not have ec2:DescribeInstances permission

Question 4

Refer to the exhibit. What is the recall of the model?

Accepted Answer

0.80. Recall is calculated as True Positives divided by the sum of True Positives and False Negatives. From the confusion matrix, True Positives = 80 and False Negatives = 20, so recall = 80 / (80 + 20) = 0.80. Option B is correct.

Answer

0.85

Answer

0.89

Answer

0.90

Question 5

A data scientist is training a deep learning model using TensorFlow on Amazon SageMaker. The training job uses a single GPU instance but the GPU utilization is low. Which action is MOST likely to improve GPU utilization?

Accepted Answer

Increase the batch size. Increasing the batch size allows the GPU to process more data in parallel per training step, which keeps the GPU compute units busier and reduces idle time. In TensorFlow on SageMaker, a small batch size can cause the GPU to finish computation quickly and then wait for the next batch to be loaded, leading to low utilization. This is the most direct way to improve GPU throughput without changing the instance or model architecture.

Answer

Use a smaller instance type

Answer

Add more features

Answer

Decrease the number of epochs

Question 6

A data scientist is training a linear regression model and observes that the training loss is low but validation loss is high. Which step should the data scientist take to address this issue?

Accepted Answer

Apply L2 regularization to the model. Option D is correct because high validation loss indicates overfitting; regularization reduces overfitting. Option A is wrong because adding more features may increase overfitting. Option B is wrong because increasing training time typically increases overfitting. Option C is wrong because reducing training data may worsen overfitting.

Answer

Increase the number of training epochs

Answer

Reduce the size of the training dataset

Answer

Add more features to the model

Question 7

A company uses Amazon SageMaker to train a deep learning model on a GPU instance. The training job is taking too long. Which action would MOST likely reduce training time?

Accepted Answer

Use a larger GPU instance type, such as p3.16xlarge. Option C is correct because using a larger GPU instance like p3.16xlarge provides significantly more GPU memory, CUDA cores, and memory bandwidth, which allows for larger batch sizes and more efficient parallel processing of matrix operations. This directly reduces training time for deep learning models by enabling faster forward and backward passes through the network, especially when the model is large enough to fully utilize the additional GPU resources.

Answer

Reduce the mini-batch size

Answer

Use distributed data parallelism across multiple smaller instances

Answer

Reduce the number of epochs

Question 8

Which TWO actions are valid ways to handle missing data in a dataset before training a machine learning model? (Select TWO.)

Accepted Answer

Delete rows with missing values. Option A is correct because deleting rows with missing values (listwise deletion) is a straightforward and valid approach when the missing data is random and the dataset is large enough that the loss of rows does not significantly reduce statistical power or introduce bias. This method avoids the need to estimate missing values and is commonly used in practice when the proportion of missing data is low.

Answer

Remove all features that have any missing values

Answer

Replace missing values with the maximum value

Answer

Ignore missing values and train the model

Question 9

A team is using SageMaker to train a deep learning model for image classification. The training job is failing with a 'CUDA out of memory' error. The team is using a p3.2xlarge instance (1 GPU, 16 GB GPU memory). The dataset consists of 256x256 RGB images. Which action is MOST likely to resolve the error without changing the instance type?

Accepted Answer

Reduce the batch size. The 'CUDA out of memory' error indicates that the GPU memory is exhausted. Reducing the batch size directly decreases the memory footprint per training step, allowing the model to fit within the 16 GB GPU memory of the p3.2xlarge instance. This is the most direct and effective fix without changing the instance type.

Answer

Increase the batch size to utilize GPU more efficiently

Answer

Enable automatic model tuning to optimize hyperparameters

Answer

Use Spot Instances to reduce cost

Question 10

A data scientist is training a model using SageMaker and wants to automatically stop training when the model stops improving. Which TWO options can be used?

Accepted Answer

Built-in early stopping in XGBoost. Built-in early stopping in XGBoost (Option B) is correct because XGBoost natively supports an `early_stopping_rounds` parameter that halts training when the validation metric stops improving for a specified number of rounds. SageMaker Debugger (Option C) is correct because it can monitor training metrics in real time and trigger a stop action via a built-in or custom rule (e.g., `VanishingGradient` or `LossNotDecreasing`) when the model stops improving, integrating with SageMaker's `StopTraining` API.

Answer

AWS Step Functions

Answer

CloudWatch Alarms

Answer

SageMaker Model Monitor

Question 11

A machine learning team is building a recommendation system for an e-commerce platform. They have user-item interaction data (clicks, purchases). They need to choose an algorithm that can capture both user and item latent factors and handle missing data. Which algorithm should they use?

Accepted Answer

Matrix factorization. Matrix factorization is the correct choice because it decomposes the user-item interaction matrix into lower-dimensional latent factors for users and items, capturing underlying patterns in preferences. It naturally handles missing data by learning from observed interactions only, making it ideal for recommendation systems with sparse data.

Answer

Linear regression

Answer

Principal component analysis (PCA)

Answer

Convolutional neural network (CNN)

Question 12

A data scientist is using Amazon SageMaker's built-in BlazingText algorithm for word2vec embeddings. The dataset is a corpus of 10 million documents. After training, the data scientist observes that the learned embeddings do not capture semantic similarity well (e.g., 'king' and 'queen' are not close). Which hyperparameter adjustment is most likely to improve the quality of embeddings?

Accepted Answer

Increase the vector dimensionality. Increasing the vector dimensionality allows the model to capture more nuanced semantic relationships and co-occurrence patterns in the data. With 10 million documents, the default dimensionality (typically 100 or 300) may be insufficient to encode the rich contextual information, so raising it (e.g., to 300 or 500) gives the model more capacity to learn high-quality embeddings where words like 'king' and 'queen' become closer in vector space.

Answer

Decrease the window size

Answer

Decrease the number of negative samples

Answer

Increase the learning rate

Question 13

A data scientist is training a classification model on an imbalanced dataset where the positive class represents only 5% of the data. Which technique would BEST address the class imbalance without discarding data?

Accepted Answer

Use SMOTE to generate synthetic samples for the minority class. SMOTE (Synthetic Minority Oversampling Technique) is the best choice because it generates synthetic samples for the minority class by interpolating between existing minority instances, effectively balancing the dataset without discarding any data. This avoids the information loss of undersampling and the overfitting risk of simple random oversampling, making it ideal for a 5% positive class scenario.

Answer

Randomly undersample the majority class

Answer

Adjust the decision threshold to 0.95

Answer

Randomly oversample the minority class with replacement

Question 14

A company uses Amazon SageMaker to train a model for fraud detection. The training data is highly imbalanced. The data scientist uses SMOTE to oversample the minority class. However, the model still has poor recall on the minority class. Which additional technique should the data scientist consider?

Accepted Answer

Use class weights in the loss function. Cost-sensitive learning assigns higher penalty to misclassifications of minority class, addressing imbalance. Option A is for feature selection, B is regularization, D is for multi-class.

Answer

One-vs-rest encoding

Answer

L1 regularization

Answer

Principal component analysis (PCA)

Question 15

A data scientist is setting up an IAM role for an Amazon SageMaker training job. The policy shown is attached to the role. The training job fails with an access denied error when trying to read the training data from s3://my-bucket/training/data.csv. What is the most likely reason?

Accepted Answer

The bucket policy on my-bucket denies access to the IAM role. Option A is correct because the IAM policy shown grants s3:GetObject access to the bucket and object, but the training job still fails with an access denied error. The most likely cause is that the bucket policy on my-bucket explicitly denies access to the IAM role, overriding the IAM policy's allow. In AWS, an explicit deny in a resource-based policy (bucket policy) takes precedence over any allow in an identity-based policy (IAM role policy), causing the access denied error despite the IAM policy appearing sufficient.

Answer

The training job is using a different IAM role

Answer

The IAM policy is missing the s3:ListBucket permission

Answer

The IAM role does not have permission to access the bucket location