MLS-C01 Modeling Practice Test 10 — 15 Questions

Question 1

A data scientist is training a deep learning model on Amazon SageMaker and notices that training is taking much longer than expected. The training job uses a single GPU instance. The model is a large transformer with millions of parameters. Which change would most likely reduce training time?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use SageMaker's distributed data parallelism with multiple GPU instances. Using data parallelism with multiple GPU instances can significantly reduce training time for large models by distributing the workload across multiple GPUs. Model parallelism is also possible but data parallelism is more common and easier to implement.

Answer

Reduce the batch size to fit in memory

Answer

Use a smaller instance type

Answer

Switch to a CPU instance

Question 2

A data scientist is building a regression model to predict housing prices. The dataset includes numerical features such as square footage, number of bedrooms, and year built, as well as categorical features such as neighborhood and roof type. Which TWO preprocessing steps are most important to apply before training a linear regression model?

Accepted Answer

One-hot encode categorical features. Linear regression requires numerical features and is sensitive to feature scales. Encoding categorical variables as numerical is necessary, and scaling numerical features ensures that no single feature dominates the model.

Answer

Apply principal component analysis (PCA) for dimensionality reduction

Answer

Remove outliers using IQR

Answer

Add interaction terms between all features

Question 3

A machine learning engineer is evaluating a multi-class classification model that predicts product categories. The model outputs probabilities for 10 classes. The engineer wants to improve the model's calibration so that the predicted probabilities reflect the true likelihood of each class. Which THREE techniques can help?

Accepted Answer

Use temperature scaling. Platt scaling and isotonic regression are common calibration methods for classification models. Temperature scaling is a variant of Platt scaling for neural networks. Using a different loss function like cross-entropy helps but is not a calibration technique per se.

Answer

Increase model complexity

Answer

Use focal loss

Question 4

A data scientist is training a text classification model using Amazon SageMaker's built-in BlazingText algorithm. The dataset contains 1 million documents. Which TWO hyperparameters are most important to tune for improving model accuracy?

Accepted Answer

Learning rate. Learning rate and number of epochs are critical hyperparameters for training neural networks like BlazingText. They control how quickly the model learns and how long it trains.

Answer

Batch size

Answer

Loss function

Answer

Type of optimizer

Question 5

A company is using Amazon SageMaker to train a large language model with billions of parameters. The training job uses multiple GPU instances in a distributed fashion. The training is converging but the loss is not decreasing as expected. The data scientist suspects that the learning rate is too high. Which technique should the data scientist use to automatically adjust the learning rate during training?

Accepted Answer

Implement learning rate scheduling with a cosine annealing schedule. Learning rate scheduling, such as a cosine annealing schedule, can automatically reduce the learning rate over time. This helps the model converge better. SageMaker's built-in algorithms support learning rate scheduling, or the user can implement it in custom training scripts.

Answer

Use a fixed learning rate and train for more epochs

Answer

Increase the batch size to reduce variance

Answer

Use gradient clipping

Question 6

A data scientist is building a recommendation system using collaborative filtering. The dataset contains user-item interactions in a sparse matrix. The model will be trained on Amazon SageMaker using the built-in Factorization Machines algorithm. Which data format should the scientist use for the training data?

Accepted Answer

RecordIO-protobuf format with sparse features. Amazon SageMaker's Factorization Machines algorithm expects input in the 'application/x-recordio-protobuf' format for sparse data, or in CSV format for dense data. For sparse data, Protobuf is recommended for performance.

Answer

CSV format with all features as columns

Answer

JSON format with nested arrays

Answer

Parquet format

Question 7

A data scientist is training a binary classification model and wants to evaluate its performance using a metric that is robust to class imbalance. Which metric should be used?

Accepted Answer

F1 score. The F1 score is the harmonic mean of precision and recall and is robust to class imbalance because it considers both false positives and false negatives. Accuracy can be misleading with imbalanced classes.

Answer

Mean squared error

Answer

Area under the ROC curve (AUC)

Answer

Accuracy

Question 8

A machine learning engineer is deploying a model to an Amazon SageMaker endpoint for real-time inference. The model requires a preprocessing step that involves tokenizing text and converting it to a numerical format. To minimize latency, where should the preprocessing logic be implemented?

Accepted Answer

Inside the SageMaker inference container using the inference.py script. To minimize latency, it's best to include the preprocessing logic inside the inference container that serves the model. This avoids additional network calls to separate preprocessing services.

Answer

Using Amazon SageMaker batch transform

Answer

As a separate AWS Lambda function called before the endpoint

Answer

On the client side before sending the request

Question 9

A data scientist is training a deep learning model on Amazon SageMaker and notices that training is taking much longer than expected. The training job uses a single GPU instance. The model is a large transformer with millions of parameters. Which change would most likely reduce training time?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Increase the number of data loader workers to load data in parallel. Low GPU utilization often indicates that the data loading pipeline is bottlenecked. Increasing the number of data loader workers can improve data throughput to the GPU, keeping it busy.

Answer

Use gradient accumulation

Answer

Use a larger instance type with more GPUs

Answer

Increase the batch size

Question 10

A data scientist is using Amazon SageMaker to train a model and wants to automatically stop the training job if the loss does not improve for a certain number of epochs. Which SageMaker feature can be used for this purpose?

Accepted Answer

Custom early stopping callback in the training script. SageMaker provides built-in early stopping via the 'StoppingCondition' parameter in the training job definition, or through custom training scripts that use callbacks. The simplest way is to set MaxRuntimeInSeconds, but for early stopping based on loss, the data scientist should implement a custom callback in the training script.

Answer

SageMaker Experiments

Answer

SageMaker Automatic Model Tuning

Answer

SageMaker Debugger

Question 11

A data scientist is training a binary classification model on an imbalanced dataset where the positive class represents 5% of the data. Which metric is most appropriate for evaluating model performance?

Accepted Answer

AUC-ROC. AUC-ROC is robust to class imbalance and measures the trade-off between true positive rate and false positive rate. Option A is wrong because accuracy can be misleading with imbalanced data. Option C is wrong because RMSE is for regression. Option D is wrong because R-squared is for regression.

Answer

Accuracy

Answer

Root Mean Squared Error (RMSE)

Answer

R-squared

Question 12

A team is using Amazon SageMaker to train a deep learning model for image classification. The training job is taking too long, and they want to reduce training time without sacrificing model accuracy. Which approach is most effective?

Accepted Answer

Use transfer learning with a pre-trained model and fine-tune on the target dataset. Transfer learning uses a pre-trained model that already has learned feature representations from a large dataset. Fine-tuning this model on the target dataset requires significantly less training time compared to training from scratch, while still achieving high accuracy. Option A is wrong because reducing batch size can slow down training and may cause convergence issues. Option B is wrong because reducing epochs can lead to underfitting and lower accuracy. Option C is wrong because reducing image resolution may remove important details, degrading model performance.

Answer

Reduce the batch size

Answer

Reduce the number of training epochs

Answer

Reduce the image resolution

Question 13

A machine learning engineer is tuning hyperparameters for a gradient boosting model using Amazon SageMaker Automatic Model Tuning. The objective metric is validation accuracy. After several tuning jobs, the best accuracy achieved is 0.85, but the engineer suspects the model is overfitting. Which hyperparameter adjustment is most likely to reduce overfitting?

Accepted Answer

Increase the regularization parameter (e.g., lambda or alpha). Increasing the regularization parameter (e.g., lambda or alpha in XGBoost) penalizes model complexity and helps reduce overfitting, making option A correct. Option B is incorrect because increasing maximum depth increases model complexity, leading to overfitting. Option C is incorrect because increasing subsample ratio (using more data per tree) can increase overfitting, while decreasing it often reduces overfitting. Option D is incorrect because increasing learning rate makes the model learn faster, which can lead to overfitting.

Answer

Increase the maximum depth of trees

Answer

Increase the subsample ratio

Answer

Increase the learning rate

Question 14

A data scientist needs to choose an algorithm for a regression problem with 50 features and 1 million training examples. The model must be interpretable and the training data fits in memory. Which algorithm is most appropriate?

Accepted Answer

Linear regression. Linear regression is the most appropriate choice because it is interpretable, handles high-dimensional data with 50 features efficiently, scales well to 1 million training examples that fit in memory, and directly addresses regression. Option A (PCA) is a dimensionality reduction technique, not a regression algorithm. Option C (XGBoost) is powerful for regression but is less interpretable due to its ensemble nature. Option D (k-NN) is non-parametric and computationally expensive at inference time, lacking interpretability.

Answer

Principal Component Analysis (PCA)

Answer

XGBoost

Answer

k-Nearest Neighbors

Question 15

A company is building a sentiment analysis model for customer reviews. The dataset includes 10,000 positive and 10,000 negative reviews. The data scientist splits the data into 70% training, 15% validation, and 15% test sets. After training, the model achieves 99% accuracy on training set but only 82% on validation set. What is the most likely issue?

Accepted Answer

The model is overfitting. The 99% training accuracy versus 82% validation accuracy indicates the model has memorized the training data but fails to generalize to unseen data, which is classic overfitting. Option D is correct. Option A is incorrect because data leakage would typically cause both training and validation accuracy to be high and similar. Option B is incorrect because the dataset is balanced (10,000 positive and 10,000 negative). Option C is incorrect because underfitting would show low accuracy on both training and validation sets.

Answer

There is data leakage from validation to training

Answer

The dataset is imbalanced

Answer

The model is underfitting