MLS-C01 Modeling Practice Test 24 — 15 Questions

Question 1

A data scientist is using SageMaker to train a random forest model. The dataset has 100 features and 1 million rows. The training job fails with a 'ResourceLimitExceeded' error. What is the MOST likely cause?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The account has reached its limit on the number of SageMaker training instances.. The 'ResourceLimitExceeded' error indicates that the account has reached its limit on the number of SageMaker training instances or vCPUs. Option A (S3 bucket region) would cause a different error, not a resource limit. Option B (GPU memory) is unlikely because random forest models typically use CPU instances. Option C (wrong algorithm) would result in an algorithm-specific error, not a resource limit. Option D correctly identifies that the account limit has been exceeded.

Answer

The S3 bucket containing the training data is not in the same region.

Answer

The instance type selected does not have enough GPU memory.

Answer

The wrong algorithm was specified for the training job.

Question 2

A company wants to use Amazon SageMaker to automatically tune hyperparameters for a XGBoost model. Which built-in SageMaker feature should be used?

Accepted Answer

SageMaker Automatic Model Tuning. SageMaker Automatic Model Tuning (AMT) is the built-in feature designed specifically to automatically search for the best hyperparameter configuration for a model. It uses Bayesian optimization (or random search) to run multiple training jobs with different hyperparameter combinations, evaluating each against a chosen objective metric. For XGBoost, you define the hyperparameter ranges (e.g., eta, max_depth, subsample) and AMT finds the optimal set, reducing manual tuning effort. This directly matches the requirement to 'automatically tune hyperparameters'.

Answer

SageMaker Debugger

Answer

SageMaker Model Monitor

Answer

SageMaker Experiments

Question 3

A data scientist trains a model using SageMaker and notices that the training loss decreases but validation loss increases after a few epochs. What is the MOST likely issue?

Accepted Answer

The model is overfitting the training data.. When training loss continues to decrease while validation loss starts to increase, the model is memorizing the training data rather than generalizing — this is the classic signature of overfitting. The divergence point indicates where the model began fitting noise in the training set at the expense of validation performance.

Answer

The learning rate is too low.

Answer

There is data leakage from validation to training.

Answer

The model is underfitting the training data.

Question 4

A company is using SageMaker to train a deep learning model with TensorFlow. The training job is running on an ml.p3.16xlarge instance. The data scientist wants to maximize GPU utilization. Which configuration should be used?

Accepted Answer

Use Pipe mode or Fast File mode with a large batch size that fits in GPU memory.. Pipe mode and Fast File mode stream training data directly from Amazon S3 into the training container without downloading the full dataset to the instance's EBS volume first, which removes the I/O bottleneck that starves GPUs on large datasets. Combining this with the largest batch size that fits in GPU memory keeps the GPU compute units saturated with matrix operations instead of waiting on data. On an ml.p3.16xlarge with 8 NVIDIA V100 GPUs, this is the standard SageMaker pattern for maximizing GPU utilization.

Answer

Use a single GPU and increase the number of epochs.

Answer

Use a CPU-only instance for training and then deploy on GPU.

Answer

Use File mode input and a small batch size.

Question 5

A data scientist is training a deep learning model on SageMaker using a custom container. The training job fails with an 'OutOfMemory' error. Which THREE actions could resolve this issue? (Choose 3.)

Accepted Answer

Use gradient accumulation to simulate larger batch sizes.. Option A is correct because gradient accumulation lets you keep an effective large batch size while processing only a few samples at a time, so the peak activation memory per step stays small and the OutOfMemory error is avoided. Option C is correct because reducing the batch size directly lowers the number of activations and gradients held in GPU memory during each forward/backward pass, which is the most common fix for OOM in deep learning training. Option D is correct because moving to an instance type with more GPU memory, such as ml.p3.16xlarge, increases the available memory pool so the same batch and model fit without exceeding the device limit. Option B is not correct because the number of epochs affects total training time, not the peak memory used in a single forward/backward pass, so it will not resolve an OOM error. Option E is not correct because increasing the learning rate changes the optimizer step size and can affect convergence, but it does not reduce the memory footprint of the model, activations, or gradients.

Answer

Reduce the number of training epochs.

Answer

Increase the learning rate.

Question 6

A data scientist is training a binary classification model on a dataset with 100 features and 10,000 samples. The model achieves 99% accuracy on the training set but only 65% on the test set. Which technique should be applied first to address this issue?

Accepted Answer

Apply L2 regularization to the model. The 99% training accuracy versus 65% test accuracy is classic overfitting, and L2 regularization (ridge) is the standard first-line remedy because it penalizes large weights and constrains model complexity. With 100 features and only 10,000 samples, the model has ample capacity to memorize noise, so adding a weight-decay penalty directly addresses the variance problem. Other options either worsen overfitting or are irrelevant to the symptom.

Answer

Reduce the size of the training dataset

Answer

Increase the number of trees in a random forest

Answer

Add more features to the model

Question 7

A machine learning team needs to deploy a model that makes real-time predictions with latency under 100 ms. The model is a deep neural network with 500 MB of parameters. Which AWS service should they use?

Accepted Answer

Amazon SageMaker real-time endpoint. Amazon SageMaker real-time endpoints are purpose-built for low-latency inference, supporting GPU instances, model caching, and autoscaling. A 500 MB deep neural network requires substantial memory and compute, and SageMaker endpoints can serve it with sub-100 ms latency using appropriate instance types (e.g., ml.g4dn, ml.p3).

Answer

AWS Glue

Answer

AWS Lambda with a container image

Answer

Amazon EMR

Question 8

A data scientist is building a time series forecasting model for daily sales data. The data exhibits strong seasonality with a weekly pattern and a yearly trend. The scientist wants to use Amazon SageMaker's built-in algorithm. Which algorithm is most appropriate?

Accepted Answer

Amazon SageMaker DeepAR. Amazon SageMaker DeepAR is a built-in algorithm specifically designed for time series forecasting. It handles complex patterns like seasonality (weekly) and trends (yearly) by using recurrent neural networks (RNNs) to model the probabilistic distribution of future values. DeepAR is well-suited for univariate and multivariate time series and can incorporate additional features, making it ideal for daily sales data with strong seasonal and trend components.

Answer

Linear Learner

Answer

K-Means

Answer

XGBoost

Question 9

A company is training a large language model on Amazon SageMaker using a single GPU instance. The training is taking too long. Which change would most likely reduce training time?

Accepted Answer

Use a larger instance with multiple GPUs and enable distributed training. Using multiple GPUs in a distributed training job can parallelize work and reduce time. Option B is wrong because increasing instance memory may not help if the bottleneck is GPU compute. Option C is wrong because decreasing batch size often increases training time due to more updates per epoch. Option D is wrong because S3 Glacier is designed for archival storage with slower retrieval, not faster access.

Answer

Increase the instance memory

Answer

Decrease the batch size

Answer

Store the training data in S3 Glacier for faster access

Question 10

A data scientist is using Amazon SageMaker to train a linear regression model. After training, the scientist notices that the model has a high bias. What is the most likely cause?

Accepted Answer

The model is too simple and underfits the data. High bias means the model consistently makes large errors because it is too simple to capture the underlying patterns in the data — this is underfitting (D). A linear regression model with high bias typically lacks the complexity or features needed to represent the true relationship, so it performs poorly on both training and validation data. The fix is to increase model complexity, add relevant features, or reduce regularization strength.

Answer

The training dataset has too many features

Answer

The model is too complex and overfits the data

Answer

The regularization parameter is too high

Question 11

A team is deploying a model for fraud detection. The dataset is highly imbalanced (99% legitimate, 1% fraudulent). They trained a logistic regression model and achieved 99% accuracy on the test set. However, the model fails to detect most fraud cases. Which metric should the team focus on to evaluate the model?

Accepted Answer

Recall. Recall measures the proportion of actual positive cases (fraud) that were correctly identified. In a highly imbalanced dataset where fraud is only 1%, a model can achieve 99% accuracy by simply predicting 'legitimate' for all cases, but it would miss all fraud cases. Recall directly addresses the goal of detecting as many fraud cases as possible, making it the most relevant metric.

Answer

Mean squared error

Answer

Precision

Answer

Accuracy

Question 12

A company wants to use Amazon SageMaker to train a model using a custom algorithm packaged in a Docker container. Which approach should they use?

Accepted Answer

Use the SageMaker SDK to create an Estimator with the image URI of the custom container. The correct approach is to use the SageMaker SDK to create an Estimator with the image URI of the custom container, as SageMaker supports bring-your-own-container for custom algorithms. Option A is incorrect because SageMaker Ground Truth is a labeling service, not for training custom algorithms. Option B is incorrect because SageMaker Autopilot automates model selection and tuning, but it does not support custom containers. Option D is incorrect because built-in algorithms are predefined and do not allow custom code.

Answer

Use SageMaker Ground Truth

Answer

Use SageMaker Autopilot

Answer

Select one of the built-in algorithms in SageMaker

Question 13

A data scientist is training a neural network on Amazon SageMaker and wants to automatically stop training if the validation loss does not improve for 5 consecutive epochs. Which feature should they use?

Accepted Answer

Implement early stopping in the training script. Early stopping is a training technique implemented within the training script itself, where the algorithm monitors validation loss and halts training if no improvement is seen for a specified number of epochs (patience). In SageMaker, this is not a built-in service feature but must be coded by the data scientist using frameworks like TensorFlow, PyTorch, or Keras callbacks. Therefore, Option A is correct because it directly addresses the requirement.

Answer

SageMaker Debugger

Answer

SageMaker Checkpointing

Answer

SageMaker Hyperparameter Tuning

Question 14

A team is building a model to predict customer churn. They have 50 features, including categorical variables with high cardinality (e.g., zip code with 10,000 unique values). Which feature engineering technique is most appropriate?

Accepted Answer

Target encoding. Target encoding replaces each high-cardinality category (e.g., each of the 10,000 zip codes) with a statistic derived from the target variable, such as the mean churn rate for that zip code. This compresses 10,000 unique values into a single numeric column, preserving predictive signal without exploding dimensionality. It is the standard technique for high-cardinality categoricals in tabular ML.

Answer

Binning zip codes into regions

Answer

Label encoding

Answer

One-hot encoding

Question 15

A data scientist is training a random forest classifier on Amazon SageMaker and wants to reduce overfitting. Which TWO actions should the scientist take? (Choose TWO.)

Accepted Answer

Limit the maximum depth of trees. Option D is correct because limiting the maximum depth of trees constrains each individual tree's ability to memorize noise in the training data, which is a standard and effective way to reduce overfitting in a random forest. Option E is correct because increasing the number of trees in a random forest reduces variance through averaging of many decorrelated trees, improving generalization without causing overfitting (more trees never hurt accuracy, they only add computation). Option A is incorrect because increasing the number of features considered per split (max_features) makes trees more correlated and more prone to overfitting, not less. Option B is incorrect because increasing the maximum depth allows trees to grow deeper and fit noise, worsening overfitting. Option C is incorrect because decreasing the number of trees reduces the averaging effect and typically increases variance, which can worsen overfitting rather than reduce it.

Answer

Increase the number of features considered per split

Answer

Increase the maximum depth of trees

Answer

Decrease the number of trees