MLS-C01 Modeling Practice Test 22 — 15 Questions

Question 1

A company is using Amazon SageMaker to train a linear learner model for predicting customer lifetime value. The target variable is right-skewed with a long tail. The data scientist applies a log transformation to the target variable and trains the model. The model achieves a low root mean squared error (RMSE) on the log scale. However, when the predictions are exponentiated back to the original scale, the RMSE is much higher. Which step should the data scientist take to improve the model's performance on the original scale?

Accepted Answer

Use a loss function that models the original distribution, such as Poisson or Tweedie. Option B (use a loss function like Poisson or Tweedie) is appropriate for non-negative skewed targets. Option A (remove outliers) may lose data. Option C (use a different algorithm) may not address the issue. Option D (increase regularization) may not help.

Answer

Increase the regularization strength

Answer

Remove outliers from the training data

Answer

Use a deep learning model instead of linear learner

Question 2

A data scientist is training a multiclass classification model to categorize support tickets into 50 categories. The dataset has 100,000 labeled tickets. The scientist uses a random forest classifier with 100 trees. The model achieves 90% accuracy on the test set, but the F1-score for some rare categories is below 0.1. The scientist wants to improve performance on rare categories without significantly reducing overall accuracy. Which approach should the scientist try?

Accepted Answer

Use class_weight='balanced' or compute custom class weights. Option A (use class weights) helps the model focus on rare classes. Option B (reduce the number of trees) may hurt overall performance. Option C (use one-vs-rest logistic regression) may not handle rare classes well. Option D (increase max_depth) could overfit.

Answer

Increase the maximum depth of trees

Answer

Reduce the number of trees to 50 to prevent overfitting

Answer

Switch to a one-vs-rest logistic regression model

Question 3

A research team is training a deep learning model for object detection using SageMaker's built-in SSD algorithm. The dataset contains 50,000 images with bounding box annotations. The team uses a single ml.p3.2xlarge instance. After 24 hours of training, the model's loss has plateaued, but the mean average precision (mAP) on validation is only 0.45. The team wants to improve mAP without increasing training time. Which action should they take?

Accepted Answer

Use a pre-trained model as the backbone (e.g., ResNet-50 pre-trained on ImageNet). Option B (use a pre-trained backbone) transfers learned features, often improving accuracy. Option A (increase batch size) may not improve mAP and could slow convergence. Option C (add more layers) increases training time. Option D (increase learning rate) may destabilize training.

Answer

Increase the learning rate by a factor of 2

Answer

Increase the batch size to 64

Answer

Add more convolutional layers to the backbone

Question 4

A data scientist is building a regression model to predict energy consumption. The dataset includes features like temperature, humidity, day of week, and holiday flags. The scientist uses a linear regression model and obtains an R-squared of 0.85 on training and 0.40 on test. The scientist suspects the model is not capturing non-linear relationships. Which approach should the scientist use to capture non-linearity?

Accepted Answer

Add polynomial features (e.g., squared terms and interactions). Option C (add polynomial features) captures non-linear relationships. Option A (increase regularization) reduces overfitting but doesn't add non-linearity. Option B (use PCA) reduces dimensionality. Option D (remove features) may lose information.

Answer

Apply PCA to the feature set

Answer

Increase L1 regularization using Lasso

Answer

Remove features with low correlation to the target

Question 5

A company is building a recommendation system using matrix factorization. The dataset has 1 million users and 100,000 items. The data scientist trains a model using SageMaker's Factorization Machines algorithm. The model achieves a root mean squared error (RMSE) of 0.95 on the test set. However, the business requires RMSE below 0.90. The data scientist has already tuned hyperparameters like number of factors and learning rate. Which additional step should the data scientist take to improve RMSE?

Accepted Answer

Add side features such as user demographics and item categories. Option D (add side features) provides more information to the model. Option A (more factors) may overfit. Option B (more iterations) may not help if converged. Option C (use linear regression) is less powerful.

Answer

Increase the number of training iterations

Answer

Increase the number of factors to 1000

Answer

Use a linear regression model instead

Question 6

A data scientist is training a binary classifier to detect network intrusions. The dataset has 1,000 features and 10 million samples, but only 0.1% are positive (intrusions). The scientist uses XGBoost with scale_pos_weight set to 100. The model achieves a recall of 0.90 and precision of 0.05 on the test set. The business requires precision of at least 0.50 while maintaining recall above 0.80. Which technique should the scientist apply?

Accepted Answer

Tune the decision threshold on validation data to maximize F1 score. Option B (post-training threshold tuning) adjusts the decision threshold to trade off precision and recall. Option A (increase scale_pos_weight) will further increase recall but decrease precision. Option C (undersample majority) can help but may reduce recall. Option D (use random forest) may not achieve required precision.

Answer

Switch to a random forest classifier with class weights

Answer

Randomly undersample the majority class to achieve 1:1 ratio

Answer

Increase scale_pos_weight to 500

Question 7

A data scientist is training a binary classifier using a large dataset with class imbalance (90% negative, 10% positive). After training a logistic regression model, the F1 score is low but accuracy is high. Which TWO actions should the data scientist take to improve model performance? (Choose 2.)

Accepted Answer

Switch to evaluation metrics such as F1 score or AUC-ROC instead of accuracy.. Option A (resample training data) and Option C (use different evaluation metric) are correct because class imbalance causes the model to be biased toward the majority class, leading to high accuracy but poor F1. Resampling (e.g., SMOTE) balances classes, and using F1 or AUC-ROC focuses on minority class performance. Option B (feature scaling) is a general preprocessing step but doesn't directly address imbalance. Option D (increase regularization) might reduce overfitting but doesn't target imbalance. Option E (add more features) may not help if the model is already biased.

Answer

Apply feature scaling to ensure all features contribute equally.

Answer

Add more features to the model to improve its capacity.

Answer

Increase the regularization parameter to reduce overfitting.

Question 8

A machine learning engineer is training a regression model to predict house prices using Amazon SageMaker. The dataset contains 10,000 samples and 50 numerical features. After training a linear regression model, the engineer notices that the training loss is low, but the validation loss is high. The engineer suspects overfitting. The dataset is already normalized. Which action should the engineer take to reduce overfitting?

Accepted Answer

Add L2 regularization (weight decay) to the loss function.. Option B (add L2 regularization) is correct because it penalizes large weights and reduces overfitting in linear models. Option A (decrease batch size) can introduce noise but doesn't directly regularize. Option C (increase learning rate) may cause divergence. Option D (reduce feature count by PCA) could help but may lose information; regularization is more direct.

Answer

Increase the learning rate to speed up convergence.

Answer

Reduce the number of features using PCA.

Answer

Decrease the mini-batch size during training.

Question 9

A data scientist is training a binary classification model on a highly imbalanced dataset where the positive class represents only 1% of the data. The model achieves 99% accuracy but only identifies 5% of the actual positives. Which metric should the data scientist use to evaluate model performance?

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual positives correctly identified by the model. With only 5% of positives detected, recall is 0.05, which directly reveals the model's failure to capture the minority class despite high accuracy. In imbalanced datasets, accuracy is misleading because the model can achieve 99% accuracy by simply predicting the majority class (negative) for all instances.

Answer

Mean squared error

Answer

Accuracy

Answer

Precision

Question 10

A team is deploying a real-time inference endpoint using Amazon SageMaker. The model is a large deep learning model that requires GPU for inference. The endpoint must handle variable traffic patterns with minimal latency. Which deployment strategy should the team use?

Accepted Answer

Use a SageMaker multi-model endpoint with GPU instance type.. B is correct because SageMaker multi-model endpoints (MMEs) allow multiple models to be hosted on a single GPU-backed endpoint, dynamically loading and unloading models from disk to GPU memory as needed. This reduces cost and cold-start latency compared to single-model endpoints, while still providing GPU acceleration for deep learning inference. MMEs are ideal for variable traffic patterns because they can scale horizontally and share GPU resources efficiently.

Answer

Deploy a single model endpoint with an auto-scaling policy.

Answer

Deploy a serverless endpoint using SageMaker Serverless Inference.

Answer

Use SageMaker Batch Transform to process requests in batches.

Question 11

A machine learning engineer is training a deep learning model using the SageMaker built-in XGBoost algorithm. The training job is taking longer than expected. The engineer notices that the training data is stored in S3 in CSV format and is 500 GB in size. The instance type is ml.c4.8xlarge with 10 instances. Which change would most likely reduce training time?

Accepted Answer

Use Pipe input mode instead of File input mode.. Pipe input mode streams data directly from S3 to the training instances without first downloading it to the local EBS volume, eliminating the I/O bottleneck of reading a 500 GB CSV file. This reduces the time spent on data loading and allows the XGBoost algorithm to begin training sooner, which is especially beneficial for large datasets.

Answer

Convert the data to Parquet format.

Answer

Increase the number of instances to 20.

Answer

Increase the size of the EBS volume attached to each instance.

Question 12

A data scientist is building a regression model to predict house prices. The dataset has 10 features, and the model shows high variance with a low bias. Which technique should the data scientist use to reduce variance?

Accepted Answer

Apply L2 regularization to the model.. L2 regularization (Ridge regression) penalizes large coefficients by adding a squared magnitude term to the loss function, which shrinks the model's weights and reduces variance without substantially increasing bias. This directly addresses the high-variance, low-bias symptom, making the model less sensitive to fluctuations in the training data.

Answer

Increase the depth of decision trees in the ensemble.

Answer

Add more features to the model.

Answer

Reduce the amount of training data.

Question 13

A company is deploying a fraud detection model using Amazon SageMaker. The model is a linear learner trained on 100 GB of data. For inference, the model receives individual transactions and must return a prediction within 100 ms. Which endpoint configuration should the team use to meet the latency requirement?

Accepted Answer

Deploy a single model endpoint using a GPU instance and enable autoscaling.. Option B is correct because a single-model endpoint on a GPU instance provides the low-latency, high-throughput inference required for real-time fraud detection. GPU instances accelerate linear learner inference by parallelizing matrix operations, enabling sub-100 ms predictions for individual transactions. Autoscaling ensures the endpoint can handle traffic spikes without degrading latency.

Answer

Use a multi-model endpoint with CPU instances.

Answer

Use a batch transform job scheduled every minute.

Answer

Deploy using SageMaker Serverless Inference.

Question 14

A data scientist is using SageMaker to train a TensorFlow model. The training script uses tf.data.Dataset to load data from S3. Training is slow because of I/O bottleneck. Which change should the data scientist make to improve I/O performance?

Accepted Answer

Use Pipe input mode for the training channel.. Option B is correct because Pipe input mode streams data directly from S3 into the training algorithm without writing to disk, eliminating the I/O bottleneck caused by downloading entire files. This is particularly effective with tf.data.Dataset, as the pipeline can consume data incrementally, reducing latency and improving throughput for large datasets.

Answer

Enable EBS optimization on the training instance.

Answer

Use SageMaker local mode for training.

Answer

Convert the dataset to RecordIO format.

Question 15

A data scientist is using Amazon SageMaker to train a model, but the training job fails with an 'Out of memory' error. The instance type is ml.p3.2xlarge. Which action should the data scientist take to resolve the issue?

Accepted Answer

Reduce the mini-batch size in the training script.. The 'Out of memory' error on a single ml.p3.2xlarge instance indicates that the GPU memory is insufficient for the current workload. Reducing the mini-batch size directly decreases the memory footprint per training step, allowing the model to fit within the available GPU memory without changing the instance type or incurring additional costs.

Answer

Use Pipe input mode.

Answer

Increase the number of instances.

Answer

Use a Spot instance.