MLS-C01 Modeling Practice Test 6 — 15 Questions

Question 1

A company is using Amazon SageMaker to deploy a model that predicts customer churn. The model was trained using a linear learner algorithm. During inference, the endpoint returns predictions that are always 0.5 (the probability of churn). What is the most likely cause?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The learning rate is set too high, causing the model to converge to the mean prediction. If the model always outputs 0.5, it suggests that the model is not learning and is stuck at the prior probability. This often happens when the learning rate is too high (causing divergence) or too low (causing slow convergence) so that the model does not update weights. The other options would cause different symptoms: data imbalance might bias towards 0 or 1, not exactly 0.5; feature scaling issues typically cause NaN or poor convergence; insufficient epochs might not converge but not necessarily give exactly 0.5.

Answer

The dataset is highly imbalanced, and the model is predicting the majority class

Answer

The model was trained with too few epochs

Answer

The input features are not normalized

Question 2

A data scientist is using Amazon SageMaker to train a deep learning model for image classification. The training job is using a single GPU instance and is taking too long. The scientist wants to reduce training time without sacrificing model accuracy. The dataset contains 100,000 images of size 256x256. Which change would most effectively reduce training time?

Accepted Answer

Use a smaller image size (e.g., 128x128). Reducing image resolution (e.g., to 128x128) significantly reduces the number of pixels and thus the computational cost per epoch, often with minimal impact on accuracy for many tasks. Using a smaller batch size increases the number of iterations but can actually slow down training. Distributed training with multiple GPUs would reduce time but the question asks for a change that does not sacrifice accuracy; distributed training can sometimes affect convergence but is generally safe. However, reducing resolution is a direct and effective method.

Answer

Reduce the batch size

Answer

Increase the learning rate

Answer

Switch to a distributed training setup with multiple GPUs

Question 3

A company uses Amazon SageMaker to host a model for real-time predictions. The model endpoint is experiencing high latency during peak hours. The data scientist wants to reduce latency without increasing cost. Which action should they take?

Accepted Answer

Enable auto-scaling for the endpoint based on latency metrics. Using SageMaker's production variants with auto-scaling can help handle traffic spikes without over-provisioning, thus managing latency and cost. Switching to a larger instance would increase cost. Reducing the number of instances would increase latency. Enabling data capture adds overhead and increases latency.

Answer

Enable data capture for the endpoint to log requests

Answer

Switch to a larger instance type

Answer

Reduce the number of instances behind the endpoint

Question 4

A data scientist is training a linear regression model on a dataset with 10 numerical features. After training, the model's R-squared value is 0.99 on the training set but only 0.60 on the test set. Which TWO of the following are appropriate actions to reduce overfitting? (Choose TWO.)

Accepted Answer

Use a subset of the most important features. Regularization (L1 or L2) penalizes large coefficients and reduces overfitting. Reducing model complexity by using fewer features or simplifying the model also helps. Adding more features would increase complexity and overfitting. Increasing the number of epochs is not relevant for linear regression (which has a closed-form solution).

Answer

Normalize the features

Answer

Add more features to the model

Answer

Increase the number of training epochs

Question 5

A company uses Amazon SageMaker to build a text classification model using a pre-trained BERT model. The dataset contains 10,000 labeled documents. The model is overfitting: training accuracy is 99%, validation accuracy is 85%. Which TWO of the following are most likely to help reduce overfitting? (Choose TWO.)

Accepted Answer

Increase the dropout rate during fine-tuning. Increasing dropout during fine-tuning adds regularization. Decreasing the learning rate can help the model converge to a better solution and prevent overfitting to the training set. Increasing batch size can sometimes regularize but is not as effective as dropout. Adding more layers increases model capacity and overfitting. Using a larger pre-trained model also increases capacity.

Answer

Add more transformer layers to the model

Answer

Increase the batch size

Answer

Use a larger pre-trained BERT model

Question 6

A data scientist is training a k-means clustering model on a dataset with 1,000 points. The scientist uses the elbow method to choose the number of clusters. The elbow plot shows a clear bend at k=4. After running k-means with k=4, the scientist wants to evaluate the quality of the clustering. Which THREE of the following are suitable internal clustering validation metrics? (Choose THREE.)

Accepted Answer

Calinski-Harabasz index. Silhouette score, Davies-Bouldin index, and Calinski-Harabasz index are all internal validation metrics that do not require ground truth labels. They measure compactness and separation. Rand index and adjusted Rand index require ground truth labels (external validation).

Answer

Adjusted Rand index

Answer

Rand index

Question 7

A data scientist is training a binary classification model on imbalanced data (95% negative, 5% positive). The model achieves 99% accuracy on the test set but fails to detect any positive cases. Which metric should the scientist focus on to evaluate model performance?

Accepted Answer

Recall. Recall (true positive rate) measures the ability to find all positive samples, which is critical for imbalanced datasets where accuracy can be misleading. Option A is wrong because accuracy is high but misleading in imbalanced data. Option C is wrong because RMSE is a regression metric, not suitable for classification. Option D is wrong because precision focuses on the accuracy of positive predictions but does not capture missed positives; recall is more important for detecting all positive cases.

Answer

Accuracy

Answer

RMSE

Answer

Precision

Question 8

A machine learning engineer is deploying a model for real-time inference using Amazon SageMaker. The model is a large ensemble that requires 8 GB of memory and 4 vCPUs. The expected traffic is 100 requests per second with a 200 ms latency requirement. Which instance configuration should they choose?

Accepted Answer

ml.c5.2xlarge (8 vCPU, 16 GB). (ml.c5.2xlarge) is correct because it offers 8 vCPUs and 16 GB memory, meeting both the compute and memory requirements while being cost-effective for CPU-based inference. Option A (ml.t2.medium) is wrong because it has only 2 vCPUs and 4 GB, insufficient for the 8 GB memory need. Option C (ml.p3.2xlarge) is wrong because it includes a GPU, which is unnecessary and overkill for this CPU-bound workload, and costs more. Option D (ml.m5.large) is wrong because it has only 2 vCPUs and 8 GB memory, lacking the required vCPUs and barely meeting memory.

Answer

ml.t2.medium (2 vCPU, 4 GB)

Answer

ml.p3.2xlarge (8 vCPU, 61 GB GPU)

Answer

ml.m5.large (2 vCPU, 8 GB)

Question 9

A data scientist is using Amazon SageMaker to train a linear regression model. The training data has 10 features, and the scientist wants to interpret the model's coefficients. Which algorithm should they use?

Accepted Answer

Amazon SageMaker Linear Learner. Amazon SageMaker Linear Learner provides interpretable coefficients, which is essential for understanding the impact of each feature in a linear regression model. Option A is wrong because XGBoost is a tree-based ensemble method that is less interpretable and does not provide linear coefficients. Option B is wrong because K-Means is an unsupervised clustering algorithm, not suited for regression. Option C is wrong because Factorization Machines are designed for high-dimensional sparse data and are not the standard choice for linear regression with 10 features.

Answer

Amazon SageMaker XGBoost

Answer

Amazon SageMaker K-Means

Answer

Amazon SageMaker Factorization Machines

Question 10

A company is building a multiclass classification model using Amazon SageMaker. The dataset has 100 classes and is highly imbalanced. The model currently achieves high accuracy on the majority classes but poor performance on minority classes. Which technique should the data scientist use to improve minority class performance?

Accepted Answer

Use class weights to penalize misclassifications of minority classes. Class weights assign a higher penalty to misclassifications of minority classes, forcing the model to pay more attention to them and improving recall for minority classes. Option A is wrong because random oversampling with replacement can cause overfitting and does not incorporate minority class importance as directly as class weights. Option B is wrong because PCA is a dimensionality reduction technique and does not address class imbalance. Option D is wrong because removing samples from the majority class (undersampling) can discard useful information and is often less effective than using class weights.

Answer

Apply random oversampling with replacement

Answer

Apply principal component analysis (PCA)

Answer

Remove samples from majority classes

Question 11

A data scientist is training an LSTM model for time series forecasting using Amazon SageMaker. The model is overfitting. Which action is LEAST likely to reduce overfitting?

Accepted Answer

Increase the number of LSTM layers. Increasing the number of LSTM layers (Option B) is least likely to reduce overfitting because it increases model complexity, which can actually worsen overfitting. In contrast, adding dropout layers (Option A), reducing the number of hidden units (Option C), and using early stopping (Option D) are all techniques that help reduce overfitting by imposing regularization or limiting training time.

Answer

Add dropout layers

Answer

Reduce the number of hidden units

Answer

Use early stopping

Question 12

A company is using Amazon SageMaker to deploy a model that predicts customer churn. The model was trained using a linear learner algorithm. During inference, the endpoint returns predictions that are always 0.5 (the probability of churn). What is the most likely cause?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

ml.m5.large. Ml.m5.large (Option C). This instance type provides 2 vCPU and 8 GB memory, which is more than sufficient for the 500 MB memory requirement. It also offers consistent performance suitable for real-time inference with 10 requests per second and 100 ms latency. Option A (ml.c5.xlarge) has 4 vCPU and 8 GB, which is over-provisioned and more expensive. Option B (ml.t2.medium) has only 4 GB memory but uses burstable CPU, which may cause latency spikes. Option D (ml.p3.2xlarge) is GPU-optimized and significantly more expensive, making it unsuitable for a CPU-bound workload. Therefore, ml.m5.large is the most cost-effective choice.

Answer

ml.c5.xlarge

Answer

ml.t2.medium

Answer

ml.p3.2xlarge

Question 13

A data scientist is using Amazon SageMaker to train a model. The training job fails with an error 'Insufficient instance capacity'. Which action should the scientist take to resolve this?

Accepted Answer

Retry with a different instance type or Availability Zone. Retrying with a different instance type or in a different Availability Zone often resolves capacity issues. Option A is incorrect because the error is about capacity, not IAM permissions. Option C is incorrect because increasing the number of instances may exacerbate the capacity issue. Option D is incorrect because reducing the training data size does not affect instance capacity.

Answer

Check IAM permissions

Answer

Increase the number of instances

Answer

Reduce the training data size

Question 14

A machine learning engineer is training a model using Amazon SageMaker. The training data is stored in S3 and is 10 TB. The engineer wants to use Pipe input mode to stream data from S3. Which algorithms support Pipe mode? (Select all that apply)

Accepted Answer

Amazon SageMaker Linear Learner. Amazon SageMaker's built-in algorithms, including Linear Learner, K-Means, XGBoost, and PCA, all support Pipe mode for streaming training data from S3. Therefore, all the listed options (A, B, C, D) are correct.

Question 15

A data scientist is training a model using Amazon SageMaker. The training job is running on GPU instances, but the GPU utilization is low. Which TWO actions could improve GPU utilization?

Accepted Answer

Use a larger instance with multiple GPUs. Using a larger instance with multiple GPUs allows more parallel processing, improving GPU utilization. Option C is correct because increasing the batch size provides more data per step, better utilizing GPU parallelism. Option A is incorrect because increasing epochs does not affect utilization per step. Option D is incorrect because switching to CPU instances would not utilize GPU. Option E is incorrect because decreasing batch size reduces parallelism and lowers GPU utilization.

Answer

Increase the number of epochs

Answer

Switch to CPU instances

Answer

Decrease the batch size