MLS-C01 Modeling Practice Test 7 — 15 Questions

Question 1

A machine learning team is using SageMaker to train a deep learning model. The training job is failing due to insufficient GPU memory. Which approach should the team take to resolve this issue without changing the model architecture?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use gradient accumulation to reduce the effective batch size per step.. Gradient accumulation allows the model to simulate a larger batch size without increasing memory usage per step, as gradients are accumulated over several smaller batches before updating weights. Option A is wrong because increasing the batch size would increase memory consumption per step, exacerbating the issue. Option C (adding more GPUs) is wrong because simply adding more GPUs to the instance does not reduce the memory usage per GPU; each GPU would still process the same batch size and run out of memory. Option D is wrong because decreasing the learning rate does not affect memory usage; it only changes the step size during optimization.

Answer

Increase the batch size.

Answer

Add more GPUs to the training instance.

Answer

Decrease the learning rate.

Question 2

A company is building a model to classify customer reviews as positive or negative. The dataset has 10,000 positive and 100 negative reviews. Which metric is most appropriate for evaluating model performance?

Accepted Answer

F1 score.. (F1 score) is most appropriate because it balances precision and recall, making it suitable for imbalanced datasets where accuracy can be misleading (e.g., a model predicting all positive would achieve 99% accuracy). Option B (Accuracy) is misleading in imbalanced scenarios. Option C (Mean squared error) is for regression, not classification. Option D (AUC-ROC) can be used but may provide an overly optimistic assessment in highly imbalanced data.

Answer

Accuracy.

Answer

Mean squared error.

Answer

AUC-ROC.

Question 3

Which TWO actions can help reduce overfitting in a neural network? (Choose 2.)

Accepted Answer

Apply L1 or L2 regularization.. Option C is correct because applying L1 or L2 regularization adds a penalty term to the loss function (the sum of absolute weights for L1, or the sum of squared weights for L2), which constrains the magnitude of the model's weights and discourages it from fitting noise in the training data, thereby reducing overfitting. Option E is correct because dropout layers randomly deactivate a fraction of neurons during each training iteration, which prevents units from co-adapting to specific training examples and forces the network to learn more robust, generalizable features. Option A is not correct because increasing the number of layers raises model capacity, making the network more expressive and typically more prone to overfitting rather than less. Option B is not correct because decreasing the learning rate only affects the optimization step size and convergence stability; it does not by itself regularize the model or reduce overfitting. Option D is not correct because increasing the training dataset size is a data-collection action, not a model-level technique for reducing overfitting, and it is not one of the marked correct answers here.

Answer

Increase the number of layers.

Answer

Decrease the learning rate.

Answer

Increase the training dataset size.

Question 4

Which THREE evaluation metrics are appropriate for a multi-class classification problem? (Choose 3.)

Accepted Answer

Confusion matrix.. A confusion matrix is correct because for multi-class classification it tabulates predicted versus actual class counts across all classes, exposing per-class true positives, false positives, and false negatives from which other metrics are derived. B accuracy is correct because it measures the proportion of correctly classified instances over all classes and is a valid, if sometimes misleading under class imbalance, aggregate metric for multi-class problems. E F1 score (macro/micro) is correct because it combines precision and recall, with macro averaging treating all classes equally and micro averaging aggregating contributions globally, both standard for multi-class evaluation. C mean squared error is not appropriate because it is a regression metric measuring squared differences between continuous predicted and actual values, not discrete class labels. D precision-recall curve is not among the marked answers because it is inherently a binary-classification tool, requiring one-vs-rest binarization to be applied to multi-class problems rather than being a direct multi-class metric.

Answer

Mean squared error.

Answer

Precision-recall curve.

Question 5

A team is deploying a real-time inference endpoint using Amazon SageMaker. The model is a large ensemble of 10 deep learning models, each 500 MB. The inference latency requirement is under 200 ms. Currently, the endpoint using a single ml.p3.2xlarge instance takes 1.5 seconds per request. Which approach is MOST likely to meet the latency requirement?

Accepted Answer

Use model parallelism to split the ensemble across multiple GPUs on a single instance.. Using model parallelism to split the ensemble across multiple GPUs on a single instance (D) is most likely to meet the latency requirement. The ensemble consists of 10 large models, and a single GPU instance takes 1.5 seconds per request, far exceeding the 200 ms target. Model parallelism distributes the models across multiple GPUs, allowing parallel execution and reducing inference time. This approach leverages the combined compute of multiple GPUs to handle the large ensemble efficiently.

Answer

Increase the batch size to process more requests per invocation.

Answer

Switch to a compute-optimized instance like c5.18xlarge.

Answer

Use SageMaker Neo to compile the model for the target instance.

Question 6

A machine learning engineer is using Amazon SageMaker to train a model. The training job fails with an out-of-memory error. The training data size is 10 GB and the instance is ml.m5.xlarge (16 GB memory). Which change is MOST likely to resolve the issue without increasing cost?

Accepted Answer

Reduce the batch size in the training script.. Reducing the batch size lowers the peak memory footprint during each forward/backward pass because fewer samples are held in GPU/CPU memory simultaneously. Since the instance has 16 GB RAM and the dataset is only 10 GB, the OOM is caused by per-batch activations and gradients, not the dataset itself. Shrinking the batch size directly reduces activation memory without changing instance type or cost.

Answer

Switch to a GPU instance like p3.2xlarge.

Answer

Use a larger instance type like ml.m5.4xlarge.

Answer

Decrease the training dataset size.

Question 7

A company is deploying a machine learning model for real-time fraud detection. The model must have low latency (under 100 ms) and high throughput. The model is an ensemble of 5 gradient boosted trees (XGBoost), each 200 MB. Which deployment strategy is MOST suitable?

Accepted Answer

Use SageMaker multi-model endpoint on a compute-optimized instance.. A SageMaker multi-model endpoint on a compute-optimized instance is most suitable because it can host multiple models behind a single endpoint, dynamically loading models into memory and unloading idle ones. This reduces the cost and latency of managing five separate endpoints while providing the CPU-optimized compute needed for XGBoost inference. It also supports high throughput by scaling horizontally and caching frequently used models.

Answer

Use AWS Lambda to invoke each model sequentially.

Answer

Deploy each model as a separate SageMaker endpoint and use a load balancer.

Answer

Deploy the ensemble on a single GPU instance with large batch processing.

Question 8

A financial services company uses Amazon SageMaker to train a model for credit risk prediction. The dataset contains 500 features and 1 million records. The target variable is binary with 20% default rate. The data scientist uses a gradient boosting algorithm (XGBoost) with default hyperparameters. After training, the model achieves 95% accuracy, but the precision for the default class is only 30%, and recall is 15%. The business requires at least 50% recall and 40% precision for the default class. The data scientist tries to adjust the decision threshold, but this does not simultaneously meet both targets. The scientist suspects that the model is not learning the default patterns well. The company also has a large dataset of unlabeled transactions that could be used. Which action should the data scientist take to improve the model?

Accepted Answer

Use the unlabeled data for semi-supervised learning with pseudo-labeling.. Using unlabeled data for pseudo-labeling can help the model learn patterns of the minority class by generating additional training examples, which is especially beneficial when the labeled dataset is imbalanced. Option A is incorrect because PCA reduces dimensionality but does not address class imbalance and may discard features important for the default class. Option C is incorrect because increasing the learning rate can cause the model to overshoot optimal minima and may lead to overfitting, not improving recall and precision for the minority class. Option D is incorrect because feature selection reduces the number of features but does not directly address class imbalance; it could even remove features that are relevant for predicting defaults.

Answer

Apply PCA to reduce dimensionality and noise.

Answer

Increase the learning rate to accelerate convergence.

Answer

Reduce the number of features using feature selection to simplify the model.

Question 9

A healthcare company is building a model to predict patient readmission within 30 days of discharge. The dataset includes 10,000 patient records with 200 features, including lab results, demographics, and historical admissions. The target variable is highly imbalanced: only 8% of patients are readmitted. The data scientist splits the data into 80% training and 20% test sets, ensuring the same proportion of readmissions in each. The scientist trains a logistic regression model and a random forest model. The logistic regression achieves 92% accuracy but recall of 10% for the readmitted class. The random forest achieves 90% accuracy but recall of 25%. The business requirement is to achieve at least 60% recall for readmissions while maintaining reasonable precision. The scientist also has access to a large collection of unlabeled patient records from other hospitals. Which strategy should the data scientist use to meet the business requirement?

Accepted Answer

Use SMOTE to oversample the minority class in the training set.. The target class is imbalanced (8% readmissions) and both models have very low recall (10% and 25%), far below the 60% requirement. SMOTE oversamples the minority class by generating synthetic examples, which helps the model learn the minority decision boundary and typically improves recall. This is the most direct strategy to meet the recall requirement without collecting new data.

Answer

Collect more labeled data from other hospitals.

Answer

Use random undersampling of the majority class in the training set.

Answer

Switch to a deep neural network with more layers.

Question 10

A data scientist is training a binary classification model on a dataset with 10,000 features. The model overfits severely. Which technique is MOST appropriate to reduce overfitting?

Accepted Answer

Apply L1 regularization (Lasso). With 10,000 features, L1 regularization (Lasso) is the most appropriate technique because it performs feature selection by shrinking less important feature coefficients to exactly zero. This directly addresses the high-dimensional overfitting by reducing model complexity and removing noise features, which is more effective than other methods for such a large feature space.

Answer

Use early stopping during training

Answer

Use PCA to reduce dimensionality

Answer

Increase the max depth of the model

Question 11

A team is training a deep learning model on Amazon SageMaker. The training job is slow because the data is stored in S3 as many small files. Which approach is MOST effective to improve training throughput?

Accepted Answer

Use SageMaker Pipe mode for training input. Using SageMaker Pipe mode streams data directly from S3, reducing startup time. Shuffling files or increasing instance count does not address the small file overhead. Using EFS would introduce latency.

Answer

Increase the number of ml.c5.xlarge instances

Answer

Shuffle the S3 objects to randomize order

Answer

Use Amazon EFS instead of S3

Question 12

A data scientist is using SageMaker to train a custom TensorFlow model. The training script reads data from S3 using TensorFlow's tf.data API. The training is bottlenecked by I/O. Which strategy would MOST effectively improve data throughput?

Accepted Answer

Use SageMaker Pipe mode and shard the S3 dataset. Using SageMaker Pipe mode with a sharded S3 dataset allows the training instances to stream data in parallel, reducing I/O bottlenecks. Increasing workers in tf.data may help but not as effectively as optimizing data ingestion. Using FSx for Lustre provides high throughput but adds cost and complexity.

Answer

Compress the data files in S3

Answer

Use Amazon FSx for Lustre as a mounted filesystem

Answer

Increase the number of parallel workers in tf.data

Question 13

A machine learning engineer is using SageMaker's built-in XGBoost algorithm for a multi-class classification problem. The training job completes but the model accuracy is low. Which THREE hyperparameters should the engineer tune to improve performance?

Accepted Answer

eta (learning rate). Option A, eta (learning rate), is correct because it controls the step size shrinkage used in each boosting round; a lower eta with more rounds often improves accuracy by allowing finer gradient descent. Option B, num_round, is correct because it sets the number of boosting iterations, and tuning it balances underfitting versus overfitting for the multi-class objective. Option D, max_depth, is correct because it governs the maximum depth of each tree, directly controlling model complexity and the ability to capture interactions among features. Option C, subsample, and Option E, colsample_bytree, are also valid XGBoost hyperparameters, but they are not among the three the question expects as the primary tuning targets for improving accuracy in this scenario.

Answer

subsample

Answer

colsample_bytree

Question 14

A company uses Amazon SageMaker to train a deep learning model for image classification. The training dataset consists of 500,000 images, each 256x256 pixels, stored in S3. The team uses a single ml.p3.2xlarge instance for training. The training time is unacceptably long (over 48 hours). The team wants to reduce training time without sacrificing model accuracy. They have already optimized the data pipeline by using SageMaker Pipe mode and sharding the S3 dataset. The model is a ResNet-50 implemented in TensorFlow. The team is considering the following options: A) Switch to a ml.p3.16xlarge instance which has 8 GPUs and more memory. B) Implement distributed data parallelism using Horovod across multiple instances. C) Use SageMaker's built-in Hyperparameter Tuning to find optimal hyperparameters. D) Reduce the image resolution to 128x128 to speed up training. Which option will MOST effectively reduce training time while maintaining accuracy?

Accepted Answer

Implement distributed data parallelism using Horovod across multiple instances. Distributed data parallelism with Horovod across multiple instances splits the training batch across GPUs on multiple nodes, dramatically reducing wall-clock training time while preserving model accuracy because each replica processes different data and gradients are all-reduced. This is the standard SageMaker approach for scaling deep learning training beyond a single instance.

Answer

Switch to a ml.p3.16xlarge instance

Answer

Reduce the image resolution to 128x128

Answer

Use SageMaker's built-in Hyperparameter Tuning

Question 15

A company has deployed a real-time inference endpoint using SageMaker for a fraud detection model. The model uses a Random Forest classifier. The endpoint receives predictions but the latency is too high. The metric shows p99 latency of 500ms, but the requirement is under 200ms. The team has already optimized the instance type to the maximum allowed by their budget. The data scientist suggests: A) Reducing the number of trees in the Random Forest model. B) Switching to a linear model like Logistic Regression. C) Enabling SageMaker's batch transform instead of real-time endpoint. D) Adding more instances to the endpoint behind a load balancer. Which option will MOST effectively reduce latency while maintaining acceptable accuracy?

Accepted Answer

Reduce the number of trees in the Random Forest model. Reducing the number of trees in a Random Forest directly reduces inference latency because each prediction requires traversing every tree and aggregating votes — fewer trees means fewer traversals. This is the most effective latency reduction while maintaining acceptable accuracy, since Random Forests are often over-provisioned with trees and a smaller ensemble can retain most predictive power.

Answer

Switch to a linear model like Logistic Regression

Answer

Enable SageMaker's batch transform

Answer

Add more instances to the endpoint