MLS-C01 Modeling Practice Test 46 — 15 Questions

Question 1

A company uses Amazon SageMaker to train a deep learning model using TensorFlow. The training job is failing with an 'OutOfMemory' error. The instance type is ml.p3.2xlarge with 16 GB GPU memory. The model has 10 million parameters. Which THREE actions should be taken to resolve the memory issue? (Choose THREE.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Reduce the batch size. Option A (Reduce the batch size) is correct because the batch size directly determines how many samples are held in GPU memory at once; lowering it reduces activation memory and is the most direct fix for an OutOfMemory error on an ml.p3.2xlarge with 16 GB of GPU memory. Option C (Enable mixed precision training) is correct because using FP16/BF16 instead of FP32 roughly halves activation and gradient memory, and NVIDIA Volta V100 GPUs in the p3 family support Tensor Cores for faster, lower-memory training. Option E (Use gradient accumulation) is correct because it lets you keep an effective large batch size by summing gradients over several small micro-batches, so each forward/backward pass fits in memory while preserving convergence behavior. Option B (Increase the number of epochs) is incorrect because it only lengthens training time and does not reduce per-step memory usage. Option D (Increase the batch size) is incorrect because it would increase GPU memory consumption and worsen the OutOfMemory error.

Answer

Increase the number of epochs

Answer

Increase the batch size

Question 2

A data scientist is performing hyperparameter optimization for a gradient boosting model using Amazon SageMaker Automatic Model Tuning. The objective metric is 'validation:logloss'. Which TWO strategies can help the tuning job converge faster? (Choose TWO.)

Accepted Answer

Use Bayesian optimization strategy. Option A is correct because SageMaker Automatic Model Tuning's Bayesian optimization strategy builds a probabilistic model of the objective function and uses it to choose the next hyperparameter combination, so it typically finds good configurations in fewer trials than uninformed search, helping the job converge faster. Option E is correct because enabling early stopping (via the SageMaker early stopping mechanism, e.g., the 'EarlyStopping' training job parameter with a training script that emits the objective metric) automatically terminates poorly performing training jobs before they run all epochs, reducing wasted compute and allowing the tuning job to reach a good result sooner. Option B is not correct because increasing the number of tuning jobs (max_jobs) simply runs more trials and generally increases total time and cost rather than making convergence faster. Option C is not correct because increasing resource limits (e.g., larger instance types or more instances) speeds up individual training jobs but does not improve the tuning algorithm's convergence toward the optimal hyperparameters. Option D is not correct because random search is an uninformed strategy that typically requires more trials than Bayesian optimization to find comparable hyperparameters, so it does not help the tuning job converge faster.

Answer

Increase the number of tuning jobs

Answer

Increase the resource limits for each training job

Answer

Use random search strategy

Question 3

A company is using Amazon SageMaker to deploy a machine learning model that predicts equipment failure. The model is a binary classifier that outputs a probability. The company wants to set a threshold such that the model correctly identifies 95% of actual failures (recall >= 0.95). The model's precision at the current threshold of 0.5 is 0.7. The data scientist evaluates the model on a test set and obtains the following confusion matrix at threshold 0.5: TP=95, FN=5, FP=40, TN=860. The total actual positives are 100. Which threshold adjustment should the data scientist make to achieve the recall goal?

Accepted Answer

Keep the threshold at 0.5. The current threshold of 0.5 already yields TP=95 and FN=5, giving recall = 95/(95+5) = 0.95, which meets the stated goal of recall >= 0.95. No threshold adjustment is needed. Decreasing the threshold would increase recall further but also increase false positives, while increasing the threshold would reduce recall below the target.

Answer

Decrease the threshold to 0.1

Answer

Increase the threshold to 0.7

Answer

Decrease the threshold to 0.3

Question 4

A data scientist is using Amazon SageMaker to train a deep learning model using a built-in algorithm. The training job uses an ml.p3.2xlarge instance and takes 10 hours to complete. The scientist wants to reduce training time without changing the algorithm or model architecture. The instance's GPU utilization is consistently at 95%, but CPU utilization is only 20%. The data input pipeline uses SageMaker Pipe mode with the 'TrainingInputMode' set to 'Pipe'. The training dataset is 200 GB in CSV format stored in S3. Which approach is most likely to reduce training time?

Accepted Answer

Use a larger instance type with more GPUs, such as ml.p3.8xlarge. GPU utilization is already at 95%, indicating the GPU is the bottleneck. Switching to a larger instance type like ml.p3.8xlarge provides four times the number of GPUs (4 vs. 1), allowing parallel processing of more data and directly reducing wall-clock training time without altering the algorithm or model architecture. The low CPU utilization (20%) confirms that the data pipeline is not a bottleneck, so I/O optimizations are unlikely to help.

Answer

Switch from Pipe mode to File mode to reduce I/O overhead

Answer

Use Pipe mode with 'S3DataType' as 'AugmentedManifestFile'

Answer

Reduce the batch size to improve GPU utilization

Question 5

A data scientist is building a binary classification model to predict customer churn. The dataset has 10,000 samples with 500 churners (positive class). Which TWO techniques should be used to address the class imbalance? (Choose 2.)

Accepted Answer

Use random undersampling of the majority class. Option C is correct because random undersampling of the majority class reduces the number of non-churner samples, rebalancing the 9,500:500 ratio so the classifier is not overwhelmed by the majority class during training. Option D is correct because SMOTE generates synthetic minority-class (churner) samples by interpolating between existing minority instances in feature space, directly increasing the positive class representation and improving the model's ability to learn the churn pattern. Option A is incorrect because a higher learning rate only affects optimization step size and can cause instability or divergence; it does nothing to correct class imbalance. Option B is incorrect because L1 regularization penalizes large weights to encourage sparsity and reduce overfitting, not to rebalance class distributions. Option E is incorrect because PCA reduces dimensionality by projecting features onto principal components, which addresses multicollinearity or high dimensionality, not class imbalance.

Answer

Use a higher learning rate during training

Answer

Use L1 regularization on the model

Answer

Use principal component analysis (PCA) to reduce dimensionality

Question 6

A data scientist is training a linear regression model on a dataset with 50 features. After training, they notice that the model performs well on training data but poorly on test data. They suspect overfitting. Which action should they take to reduce overfitting?

Accepted Answer

Add L2 regularization (Ridge regression). L2 regularization (Ridge regression) adds a penalty term proportional to the square of the magnitude of the coefficients to the loss function. This discourages the model from fitting the noise in the training data by shrinking the weights, which reduces variance and mitigates overfitting, improving generalization to the test data.

Answer

Use a larger learning rate

Answer

Add more features to the model

Answer

Increase the number of training epochs

Question 7

A company is using SageMaker built-in object detection algorithm to detect defects in manufacturing images. The model is trained on 10,000 labeled images and achieves 95% accuracy. However, in production, the model misclassifies many defective items as non-defective (false negatives). The business requires recall > 90% for the defect class. Which action should they take?

Accepted Answer

Adjust the decision threshold of the model to increase recall at the expense of precision. For a binary classifier, the decision threshold determines the trade-off between precision and recall. Lowering the threshold for the defect class causes more items to be classified as defective, which increases recall (fewer false negatives) at the cost of precision. Since the business requires recall > 90% for the defect class, adjusting the threshold is the fastest, lowest-cost action that directly addresses the metric.

Answer

Use a different algorithm such as semantic segmentation

Answer

Use SageMaker's Automatic Model Tuning to find better hyperparameters

Answer

Retrain the model with more images of non-defective items

Question 8

A media company uses SageMaker to train a neural network for content recommendation. The model uses embeddings for users and items. Training is slow and they want to reduce time. The dataset has 10 million users and 1 million items. They have a cluster of 8 p3.16xlarge instances. Which strategy is most likely to reduce training time?

Accepted Answer

Use SageMaker's model parallelism to split the embedding layers across GPUs. SageMaker's model parallelism splits the embedding layers across GPUs, which is essential when the embedding table is too large to fit into the memory of a single GPU. With 10 million users and 1 million items, even with a modest embedding dimension of 256, the embedding layer alone can exceed 10 GB, causing memory bottlenecks that slow training. Model parallelism distributes these large parameters across multiple GPUs, reducing per-GPU memory pressure and enabling larger batch sizes, which directly reduces training time.

Answer

Use data parallelism to replicate the model on each GPU and synchronize gradients

Answer

Reduce the embedding dimension from 256 to 64

Answer

Use a smaller batch size to fit on each GPU

Question 9

A data scientist is training a binary classifier using logistic regression. The dataset has 100 features and 1 million samples. After training, the model achieves AUC of 0.85 on the test set. The business wants to understand which features contribute most to predictions. Which technique should the data scientist use?

Accepted Answer

Use the coefficients of the logistic regression model as feature importance. Logistic regression is inherently interpretable: each feature has a coefficient that represents the change in log-odds of the positive class per unit change in that feature, holding others constant. With 100 features and 1 million samples, the coefficients are stable and directly usable as feature importance. This is the simplest and most appropriate technique for understanding which features contribute most to predictions.

Answer

Use t-SNE to visualize feature importance

Answer

Use a random forest model and its feature importance attribute

Answer

Use Principal Component Analysis (PCA) to find important components

Question 10

A research lab is training a large language model (LLM) on SageMaker using PyTorch. The model has 1 billion parameters and does not fit on a single GPU. They have access to a cluster of 16 p4d.24xlarge instances (each with 8 A100 GPUs). They need to train the model with minimal changes to the training script. Which SageMaker feature should they use?

Accepted Answer

SageMaker's model parallelism with automatic partitioning. SageMaker's model parallelism library automatically partitions a large model across multiple GPUs and instances, and it integrates with PyTorch via a simple wrapper (smdistributed.modelparallel.torch) that requires minimal changes to the training script. With 1B parameters and 128 A100 GPUs available, model parallelism is the correct choice because the model does not fit on a single GPU.

Answer

SageMaker's distributed data parallelism with Horovod

Answer

Use SageMaker's built-in BlazingText algorithm

Answer

SageMaker's managed spot training with checkpointing

Question 11

A data scientist is building a time series forecasting model for monthly sales. The data shows strong seasonality with a yearly pattern. They plan to use Amazon Forecast. Which algorithm should they choose?

Accepted Answer

DeepAR+. DeepAR+ is purpose-built for time series forecasting with strong seasonality, as it uses recurrent neural networks (RNNs) to capture complex temporal dependencies and automatically models seasonal patterns like yearly cycles. Amazon Forecast natively supports DeepAR+ for such use cases, making it the optimal choice over general-purpose or non-forecasting algorithms.

Answer

XGBoost

Answer

K-means clustering

Answer

Linear regression

Question 12

A data scientist is training a binary classification model on an imbalanced dataset (95% negative class, 5% positive class). The model currently achieves 94% accuracy but a recall of only 0.10 on the positive class. Which TWO strategies should the data scientist consider to improve recall without significantly sacrificing precision? (Choose 2.)

Accepted Answer

Assign higher class weights to the positive class in the loss function.. Option C is correct because assigning higher class weights to the positive class in the loss function makes errors on the minority (positive) class more costly, directly pushing the model to increase recall on that class while still balancing precision through the weighted objective. Option E is correct because SMOTE (Synthetic Minority Over-sampling Technique) generates synthetic positive-class examples via k-nearest-neighbor interpolation, enlarging the minority class and giving the model more positive signal to learn from, which typically raises positive-class recall without the severe precision loss of naive duplication. Option A is not ideal here because undersampling the majority class to 5% discards ~90% of the negative data, which can hurt precision and overall generalization. Option B is unrelated: increasing regularization strength reduces overfitting but does not address class imbalance and can even lower recall by shrinking the model's fit. Option D is also unrelated: adding layers increases capacity but does not correct the class imbalance and may worsen overfitting on the minority class.

Answer

Undersample the majority class to match the minority class size.

Answer

Increase the regularization strength to reduce overfitting.

Answer

Use a deeper neural network with more layers.

Question 13

A machine learning engineer is training a linear regression model on a dataset where the relationship between the independent and dependent variables is highly non-linear. The engineer wants to use Amazon SageMaker's built-in Linear Learner algorithm but needs to capture the non-linearity. Which approach should the engineer take to best achieve this while using the Linear Learner algorithm?

Accepted Answer

Apply feature engineering to create polynomial and interaction terms before training.. Linear Learner models a linear relationship between input features and the target. To capture non-linearity, the feature space must be transformed so that a linear model in the transformed space represents a non-linear function in the original space. Creating polynomial and interaction terms is a direct way to achieve this. Other options either do not change the model's linear nature or deviate from the requirement to use Linear Learner.

Answer

Increase the number of epochs and set the learning rate to a very small value.

Answer

Use the 'auto' mode for feature normalization and set the predictor type to 'regressor'.

Answer

Switch to the built-in XGBoost algorithm, which inherently models non-linearity.

Question 14

A data scientist is training a gradient boosting model using the SageMaker built-in XGBoost algorithm on a tabular dataset. The model performs well on the training set but poorly on the validation set. The data scientist suspects overfitting. Which hyperparameter tuning strategy should be used to reduce overfitting while maintaining reasonable training time?

Accepted Answer

Decrease the value of max_depth and increase the value of gamma.. Overfitting in XGBoost is controlled by constraining model complexity. Limiting max_depth reduces the depth of each tree, and raising gamma imposes a minimum loss reduction for splits, acting as a pruning threshold. These regularization parameters directly address overfitting without requiring excessive additional training rounds. Adjusting subsample or colsample_bytree can also help, but the combination of shallower trees and stronger gamma is the most direct and reliable fix.

Answer

Increase the value of subsample and set colsample_bytree to 1.0.

Answer

Decrease the value of eta and decrease the value of gamma.

Answer

Increase the value of max_depth and increase num_round.

Question 15

A financial institution is training a fraud detection model using Amazon SageMaker. The dataset is extremely imbalanced, with only 0.1% of transactions being fraudulent. The data scientist wants to maximize the model's ability to identify fraudulent transactions while minimizing false positives. Which evaluation metric should the data scientist use to optimize the model's performance?

Accepted Answer

Area Under the Precision-Recall Curve (AUC-PR). For highly imbalanced datasets, AUC-PR is preferred because it focuses on the minority class and the trade-off between precision and recall. AUC-ROC can be misleading due to the large number of true negatives. Accuracy is not suitable. While F1 score is useful, it evaluates a single threshold and does not capture the overall performance across thresholds as AUC-PR does.

Answer

Accuracy

Answer

F1 score with a threshold of 0.5

Answer

Area Under the ROC Curve (AUC-ROC)