MLS-C01 Modeling Practice Test 19 — 15 Questions

Question 1

A retail company uses Amazon SageMaker to train a model for product demand forecasting. The dataset contains daily sales data for 10,000 products over 3 years. The data includes features like price, promotions, holidays, and seasonality. The data scientist uses a linear regression model and gets an RMSE of 50 units. However, the business requires more accurate forecasts, especially for products with high variability. The scientist notices that the residuals show a pattern: the model underestimates demand during promotional periods. Which approach should the scientist take to improve the model?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Add interaction features between promotion and other variables.. Adding interaction features between promotion and other variables allows the model to capture the specific effect of promotions on demand, which the linear regression currently underestimates. Option B (more data) may help but won't directly address the structural bias; Option C (LSTM) might be overkill and not directly solve the underestimation during promotions; Option D (removing promotion features) would worsen the problem by discarding valuable information.

Answer

Collect more historical data for training.

Answer

Use a deep learning model like LSTM.

Answer

Remove promotion features to simplify the model.

Question 2

A data scientist is training a binary classification model on a dataset with 10,000 features. The model overfits severely. Which technique is MOST appropriate to reduce overfitting?

Accepted Answer

Apply L1 regularization (Lasso). With 10,000 features, L1 regularization (Lasso) is the most appropriate technique because it performs feature selection by shrinking less important feature coefficients to exactly zero. This directly addresses the high-dimensional overfitting by reducing model complexity and removing noise features, which is more effective than other methods for such a large feature space.

Answer

Use early stopping during training

Answer

Use PCA to reduce dimensionality

Answer

Increase the max depth of the model

Question 3

A team is training a deep learning model on Amazon SageMaker. The training job is slow because the data is stored in S3 as many small files. Which approach is MOST effective to improve training throughput?

Accepted Answer

Use SageMaker Pipe mode for training input. Using SageMaker Pipe mode streams data directly from S3, reducing startup time. Shuffling files or increasing instance count does not address the small file overhead. Using EFS would introduce latency.

Answer

Increase the number of ml.c5.xlarge instances

Answer

Shuffle the S3 objects to randomize order

Answer

Use Amazon EFS instead of S3

Question 4

A machine learning engineer is using SageMaker to train an XGBoost model on a dataset with a severe class imbalance (1:1000). The goal is to maximize recall on the minority class. Which hyperparameter tuning strategy is MOST appropriate?

Accepted Answer

Set scale_pos_weight to the ratio of negative to positive samples. XGBoost's 'scale_pos_weight' parameter can be set to the ratio of negative to positive instances to help the model focus on the minority class. Adjusting max_delta_step or subsample may help but are secondary. Setting objective to 'binary:logistic' is default, not addressing imbalance.

Answer

Set max_delta_step to a high value

Answer

Increase subsample ratio to 1.0

Answer

Set objective to 'binary:logistic' and tune max_depth

Question 5

A data scientist is evaluating a regression model. The RMSE on the training set is 2.5, and on the test set is 2.7. The R² on the test set is 0.98. What does this indicate?

Accepted Answer

The model generalizes well with no severe overfitting. The model has low error and high R² on both sets, indicating good generalization without significant overfitting. The small difference between training and test RMSE suggests no severe overfitting.

Answer

The model has high bias

Answer

The model is underfitting because R² is too high

Answer

The model is overfitting because RMSE is lower on training data

Question 6

A company uses SageMaker to host a real-time inference endpoint for a classification model. The endpoint receives traffic spikes that cause high latency. The team wants a solution that automatically scales based on demand while keeping costs low. Which approach is BEST?

Accepted Answer

Enable automatic scaling on the endpoint using Application Auto Scaling. SageMaker endpoints support automatic scaling with Application Auto Scaling based on custom metrics like 'InvocationsPerInstance' or 'SageMakerVariantInvocationsPerInstance'. Provisioned concurrency is not available for SageMaker endpoints. Spot instances are not recommended for real-time endpoints due to interruptions. Multi-model endpoints help but scaling is still needed.

Answer

Use provisioned concurrency for the endpoint

Answer

Use a multi-model endpoint to serve multiple models

Answer

Deploy the endpoint on Spot Instances

Question 7

A data scientist is using SageMaker to train a custom TensorFlow model. The training script reads data from S3 using TensorFlow's tf.data API. The training is bottlenecked by I/O. Which strategy would MOST effectively improve data throughput?

Accepted Answer

Use SageMaker Pipe mode and shard the S3 dataset. Using SageMaker Pipe mode with a sharded S3 dataset allows the training instances to stream data in parallel, reducing I/O bottlenecks. Increasing workers in tf.data may help but not as effectively as optimizing data ingestion. Using FSx for Lustre provides high throughput but adds cost and complexity.

Answer

Compress the data files in S3

Answer

Use Amazon FSx for Lustre as a mounted filesystem

Answer

Increase the number of parallel workers in tf.data

Question 8

A data scientist is building a binary classifier and obtains the following confusion matrix on the test set: TP=80, FP=20, TN=70, FN=30. What is the precision?

Accepted Answer

0.8. Precision = TP / (TP+FP) = 80/(80+20)=0.8. Recall = TP/(TP+FN)=80/110≈0.727. Accuracy = (80+70)/200=0.75. F1 = 2*(0.8*0.727)/(0.8+0.727)≈0.762.

Answer

0.727

Answer

0.75

Answer

0.762

Question 9

A team is training a large language model using SageMaker's distributed training. They notice that the training loss is not decreasing after the first few epochs. Which action is MOST likely to resolve this issue?

Accepted Answer

Reduce the learning rate. When training loss plateaus or does not decrease after the first few epochs, a common cause is that the learning rate is too high, causing the optimizer to overshoot minima. Reducing the learning rate helps the model converge. Increasing the batch size (option A) mainly affects gradient variance and training speed but does not address an overly large step size. Adding L2 regularization (option B) helps prevent overfitting but does not resolve a high learning rate. Switching from Adam to SGD (option D) may not help because Adam typically adapts learning rates per parameter; if the base learning rate is too high, both optimizers can struggle. Therefore, reducing the learning rate is the most direct and effective action.

Answer

Increase the batch size

Answer

Add L2 regularization

Answer

Switch from Adam to SGD optimizer

Question 10

A company is deploying a model that predicts customer churn. The model's recall for the churn class is 0.9, but precision is 0.4. The business cost of false positives is high. Which strategy would MOST likely improve precision without significantly harming recall?

Accepted Answer

Increase the decision threshold for the churn class. Adjusting the decision threshold to require a higher probability before predicting churn can reduce false positives (increase precision) but may lower recall. The goal is to find a threshold that balances both. Using more aggressive regularization or different algorithms may not directly control the trade-off.

Answer

Collect more data for the churn class

Answer

Use a different algorithm such as Random Forest

Answer

Decrease the decision threshold for the churn class

Question 11

A data scientist is training a linear regression model and wants to handle multicollinearity among features. Which TWO actions are appropriate?

Accepted Answer

Use Ridge regression (L2 regularization). Ridge regression (L2) adds a penalty that can reduce the impact of correlated features. Removing one of the correlated features directly addresses multicollinearity. Lasso (L1) may also help but is less effective for groups of correlated features. Scaling features does not remove collinearity. Adding interaction terms increases multicollinearity.

Answer

Add interaction terms between features

Answer

Use Lasso regression (L1 regularization)

Answer

Scale all features to have zero mean and unit variance

Question 12

A machine learning engineer is using SageMaker's built-in XGBoost algorithm for a multi-class classification problem. The training job completes but the model accuracy is low. Which THREE hyperparameters should the engineer tune to improve performance?

Accepted Answer

eta (learning rate). XGBoost hyperparameters: 'num_round' (number of boosting rounds), 'eta' (learning rate), and 'max_depth' (tree depth) are key for improving accuracy. 'subsample' can help but is less direct. 'min_child_weight' also important but these three are most common. 'colsample_bytree' is for feature subsampling.

Answer

subsample

Answer

colsample_bytree

Question 13

A data scientist is evaluating a binary classification model. The model's AUC-ROC is 0.95. Which TWO statements are true?

Accepted Answer

The model has excellent discriminative ability. AUC-ROC measures the model's ability to distinguish between classes across all thresholds. A high AUC (close to 1) indicates good performance. AUC-ROC is threshold-independent. It does not directly indicate accuracy or calibration.

Answer

The model has no false positives

Answer

The model is well-calibrated

Answer

The model's accuracy is at least 95%

Question 14

A company uses Amazon SageMaker to train a deep learning model for image classification. The training dataset consists of 500,000 images, each 256x256 pixels, stored in S3. The team uses a single ml.p3.2xlarge instance for training. The training time is unacceptably long (over 48 hours). The team wants to reduce training time without sacrificing model accuracy. They have already optimized the data pipeline by using SageMaker Pipe mode and sharding the S3 dataset. The model is a ResNet-50 implemented in TensorFlow. The team is considering the following options: A) Switch to a ml.p3.16xlarge instance which has 8 GPUs and more memory. B) Implement distributed data parallelism using Horovod across multiple instances. C) Use SageMaker's built-in Hyperparameter Tuning to find optimal hyperparameters. D) Reduce the image resolution to 128x128 to speed up training. Which option will MOST effectively reduce training time while maintaining accuracy?

Accepted Answer

Implement distributed data parallelism using Horovod across multiple instances. Using multiple instances with Horovod for distributed data parallelism can scale training linearly with the number of GPUs, significantly reducing time. A larger single instance (ml.p3.16xlarge) provides 8 GPUs but still limited by single instance. Hyperparameter tuning does not directly reduce training time. Reducing resolution may lose accuracy.

Answer

Switch to a ml.p3.16xlarge instance

Answer

Reduce the image resolution to 128x128

Answer

Use SageMaker's built-in Hyperparameter Tuning

Question 15

A data scientist is working on a regression problem to predict house prices. The dataset has 80 features, including categorical variables with high cardinality (e.g., zip code with 10,000 unique values). The target variable is log-transformed. The data scientist trains a linear regression model and obtains an R² of 0.45 on the test set. To improve performance, the data scientist considers: A) Applying one-hot encoding to all categorical features and using Ridge regression. B) Using target encoding for high-cardinality features and using a tree-based model like XGBoost. C) Removing all categorical features and using polynomial features for numerical features. D) Using principal component analysis (PCA) on all features before training a linear model. Which approach is MOST likely to improve the model's performance?

Accepted Answer

Target encoding + XGBoost. Target encoding efficiently handles high-cardinality features, and tree-based models like XGBoost can capture non-linear relationships and interactions, likely improving R². One-hot encoding would create too many features, causing sparsity. Removing categories loses information. PCA may discard important information.

Answer

Remove categorical features and use polynomial features

Answer

One-hot encoding + Ridge regression

Answer

PCA on all features before linear regression