MLS-C01 Modeling Practice Test 29 — 15 Questions

Question 1

A data scientist is training a binary classifier on an imbalanced dataset where the positive class represents only 2% of the data. The model achieves 99% accuracy but only identifies 5% of actual positives. Which metric should the scientist use to evaluate the model's ability to detect the positive class?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual positives correctly identified, which is the key concern here. Accuracy is misleading due to class imbalance.

Answer

Accuracy

Answer

F1-score

Answer

Precision

Question 2

A team is training a large deep learning model on Amazon SageMaker. The training job is taking too long and they want to reduce training time without changing the model architecture. Which action is MOST effective?

Accepted Answer

Use a GPU instance (e.g., p3.2xlarge) for training. Deep learning training is dominated by matrix multiplications that are highly parallelizable, and GPUs are designed with thousands of cores optimized for these operations. Switching from a CPU-based instance to a GPU instance like p3.2xlarge can reduce training time by an order of magnitude without changing the model architecture. This is the most direct and effective action for accelerating deep learning training.

Answer

Switch to a compute-optimized instance like c5.4xlarge

Answer

Increase the batch size and learning rate proportionally

Answer

Use SageMaker Automatic Model Tuning with hyperparameter optimization

Question 3

A machine learning engineer needs to deploy a model that makes real-time predictions with latency under 100ms. The model is a small ensemble of decision trees. Which AWS service is MOST suitable?

Accepted Answer

Amazon SageMaker endpoint. Amazon SageMaker endpoints are purpose-built for real-time inference, providing low-latency (often single-digit to tens of milliseconds) HTTPS endpoints backed by autoscaling instances. For a small ensemble of decision trees, the model is lightweight enough to meet sub-100ms latency easily on a SageMaker endpoint with appropriate instance sizing. SageMaker also supports multi-model endpoints and serverless inference for cost efficiency.

Answer

Amazon EMR with Spark Streaming

Answer

AWS Glue

Answer

AWS Lambda with custom container

Question 4

A data scientist is using Amazon SageMaker to train a model. The training data is stored in an S3 bucket encrypted with AWS KMS. During training, the job fails with an access denied error. What is the MOST likely cause?

Accepted Answer

The SageMaker execution role lacks kms:Decrypt permission for the KMS key. SageMaker needs permission to use the KMS key to decrypt the data; the execution role must have kms:Decrypt permissions.

Answer

The training instance type does not support encryption

Answer

The training data is not in the same region as the SageMaker notebook

Answer

The S3 bucket policy does not allow SageMaker to list objects

Question 5

A team is training a deep learning model using TensorFlow on a single GPU instance in SageMaker. The GPU utilization is below 30%. Which change will MOST improve GPU utilization?

Accepted Answer

Increase the batch size. Low GPU utilization (below 30%) usually means the GPU is starved for work because the batch size is too small to saturate its compute units, so increasing the batch size gives the GPU more data to process per step and raises utilization. This is the most direct, single-instance change that improves GPU efficiency without altering the training architecture or requiring distributed infrastructure. Reducing epochs or switching to CPU does not address utilization, and distributed training adds complexity without fixing the underlying per-GPU starvation.

Answer

Reduce the number of epochs

Answer

Use SageMaker Distributed Training with multiple GPUs

Answer

Switch to a CPU instance

Question 6

A machine learning team is using Amazon SageMaker to build a regression model. The target variable is heavily right-skewed with a long tail. Which data transformation should the team apply to the target variable before training?

Accepted Answer

Log transformation. A heavily right-skewed target variable with a long tail violates the normality assumption of many regression models and can cause the model to be dominated by extreme values. A log transformation compresses the tail and makes the distribution more symmetric, which stabilizes variance and improves linear model performance. This is the standard transformation for right-skewed targets in regression.

Answer

One-hot encoding

Answer

Min-max scaling

Answer

Standardization (z-score)

Question 7

A data scientist is using Amazon SageMaker to train a model with a large dataset that does not fit into memory on a single instance. The training algorithm supports distributed training. Which approach should the scientist use to train the model efficiently?

Accepted Answer

Use SageMaker Pipe mode to stream data directly from S3. SageMaker Pipe mode streams data from S3 directly to the training algorithm without writing to disk, enabling processing of large datasets beyond memory.

Answer

Use SageMaker File mode and increase the instance volume size

Answer

Use Amazon EMR to preprocess data and then train on a smaller sample

Answer

Split the data into smaller files and use multiple training jobs sequentially

Question 8

A machine learning engineer is monitoring a deployed model on SageMaker and notices that the prediction latency is increasing over time. The model is a linear regression with a small number of features. Which is the MOST likely cause?

Accepted Answer

The inference code has a memory leak. A linear regression model with a small number of features has a tiny memory and compute footprint, so the model itself is not the bottleneck. Increasing prediction latency over time on a long-running SageMaker endpoint is a classic symptom of a memory leak in the inference code — for example, accumulating request data in a global list, not releasing tensors, or leaking file handles. As memory fills, garbage collection and swapping increase, degrading latency progressively.

Answer

The number of features is too large

Answer

The CPU utilization is too low

Answer

The model is overfitting to recent data

Question 9

A company uses a SageMaker endpoint for real-time inference. They need to ensure high availability during deployment updates. Which THREE steps achieve this? (Choose 3)

Accepted Answer

Use blue/green deployment with a new endpoint configuration. Option B is correct because blue/green deployment creates a new endpoint configuration and fleet, then shifts traffic only after the new instances are healthy, so the old endpoint keeps serving during the update and rollback is possible. Option C is correct because configuring multiple instances behind the endpoint provides redundancy across Availability Zones, so the loss of one instance does not take down real-time inference. Option E is correct because SageMaker's Canary and Linear traffic shifting modes gradually move a percentage of invocations to the new variant while the old variant continues serving, maintaining availability throughout the update. Option A is wrong because a single instance is a single point of failure and cannot provide high availability. Option D is wrong because deleting the old endpoint before creating the new one causes downtime, which is the opposite of high availability.

Answer

Use a single instance to save costs

Answer

Delete the old endpoint before creating the new one

Question 10

A data scientist is building a classification model and wants to evaluate its performance. Which TWO metrics are appropriate for a multi-class classification problem? (Choose 2)

Accepted Answer

Recall. Recall (B) is correct because it measures the proportion of actual instances of each class that the model correctly identified, and in multi-class settings it can be computed per class and then averaged (macro, micro, or weighted), making it a standard classification metric. Precision (C) is also correct because it measures the proportion of predicted positives for a class that are truly that class, and like recall it extends naturally to multi-class problems through per-class and averaged variants. MAE (A), R-squared (D), and RMSE (E) are regression metrics that quantify the magnitude of numeric prediction errors or explained variance, so they are not appropriate for evaluating a categorical multi-class classifier.

Answer

Mean Absolute Error (MAE)

Answer

R-squared

Answer

Root Mean Square Error (RMSE)

Question 11

A company is training a deep learning model on Amazon SageMaker. The training job is taking a long time and the data scientist suspects that the model is overfitting. Which of the following actions can help reduce overfitting and improve generalization?

Accepted Answer

Add dropout layers to the model architecture.. Dropout is a regularization technique that randomly deactivates a fraction of neurons during each training step, forcing the network to learn redundant, distributed representations rather than memorizing training samples. This directly combats overfitting by reducing co-adaptation of neurons and effectively approximating an ensemble of subnetworks at inference time. In SageMaker training jobs, adding dropout layers (e.g., via the framework's model definition) is a standard, low-cost architectural change that improves generalization without altering the data pipeline.

Answer

Increase the batch size used during training.

Answer

Increase the number of training epochs.

Answer

Remove regularization terms from the loss function.

Question 12

A data scientist is building a binary classification model to predict customer churn. The dataset is highly imbalanced, with only 5% of customers churning. The scientist evaluates several models using accuracy, precision, recall, and F1 score. Which metric is most appropriate for comparing model performance in this scenario?

Accepted Answer

F1 score. The F1 score is the harmonic mean of precision and recall, making it the most appropriate metric for imbalanced binary classification because it balances both false positives and false negatives. In a dataset with only 5% churn, accuracy is misleading (a model predicting 'no churn' always achieves 95% accuracy), while precision or recall alone ignore the other type of error. F1 provides a single score that reflects performance on the minority class, which is typically the class of interest in churn prediction.

Answer

Accuracy

Answer

Precision

Answer

Recall

Question 13

A company is using Amazon SageMaker to train a linear regression model. The data scientist notices that the training loss is decreasing but the validation loss has started to increase after a few epochs. What is the most likely cause?

Accepted Answer

The model is overfitting the training data.. A diverging pattern — training loss continuing to fall while validation loss rises — is the textbook signature of overfitting: the model is memorizing training-specific noise rather than learning generalizable patterns. In SageMaker linear regression this typically appears after several epochs once the model has enough capacity relative to the data.

Answer

The model is underfitting the training data.

Answer

There is data leakage from the validation set into the training set.

Answer

The learning rate is too high.

Question 14

A data scientist is using Amazon SageMaker built-in XGBoost algorithm to train a regression model. The training job completes successfully but the model performance on the test set is poor, with high bias. Which hyperparameter adjustment is most likely to help reduce bias?

Accepted Answer

Increase the max_depth parameter.. High bias means the model is underfitting, so increasing max_depth allows XGBoost to build deeper trees that capture more complex interactions and reduce bias. Deeper trees increase variance, but when bias is the diagnosed problem, adding capacity is the correct direction. This is the standard bias-variance tradeoff adjustment for a boosted-tree model.

Answer

Reduce the num_round parameter.

Answer

Increase the gamma parameter.

Answer

Decrease the max_depth parameter.

Question 15

A data scientist is training a neural network on Amazon SageMaker. The network has many layers and the training is very slow. The scientist suspects that the gradients are vanishing. Which technique is most specifically designed to mitigate the vanishing gradient problem?

Accepted Answer

Use batch normalization.. Batch normalization helps by normalizing the activations, which reduces the problem of vanishing/exploding gradients. Dropout is for regularization. Data augmentation increases data. Gradient clipping deals with exploding gradients, not vanishing.

Answer

Use gradient clipping.

Answer

Use data augmentation.

Answer

Use dropout layers.