MLS-C01 Modeling Practice Test 30 — 15 Questions

Question 1

A data scientist is training a binary classification model using Amazon SageMaker's XGBoost. The dataset is highly imbalanced (99% negative class, 1% positive class). The data scientist wants to maximize the F1-score. Which parameter adjustment is most appropriate?

Accepted Answer

Set scale_pos_weight to 99. Setting scale_pos_weight to 99 is the most appropriate adjustment because it directly addresses class imbalance by assigning a higher weight to the minority (positive) class during training. In XGBoost, scale_pos_weight controls the balance of positive and negative weights, typically set as sum(negative instances) / sum(positive instances), which here is 99/1 = 99. This forces the model to penalize misclassifications of the positive class more heavily, thereby improving recall and F1-score.

Answer

Set max_depth to 10

Answer

Set eta to 0.01

Answer

Set subsample to 0.5

Question 2

A data scientist is using Amazon SageMaker to train a model on a dataset that contains both numerical and categorical features. The categorical features have high cardinality (e.g., postal codes, product IDs). Which feature engineering approach is most suitable for handling these high-cardinality categorical features in a tree-based model?

Accepted Answer

Use label encoding. Label encoding is suitable for tree-based models because these models split on feature values and can handle ordinal relationships implicitly. Unlike linear models, tree-based models do not assume any distance metric between categories, so label encoding avoids the dimensionality explosion of one-hot encoding while preserving the ability to capture splits based on high-cardinality features.

Answer

One-hot encode the categorical features

Answer

Apply target encoding

Answer

Apply binary encoding

Question 3

A data scientist is training a neural network for a multi-class classification problem with 100 classes. The model uses a softmax output layer and cross-entropy loss. During training, the loss decreases steadily but the accuracy on the validation set plateaus early. Which of the following is the most likely cause?

Accepted Answer

The model is overfitting the training data. When the validation accuracy plateaus early while training loss continues to decrease, it indicates that the model is memorizing the training data rather than learning generalizable patterns. This is classic overfitting, where the softmax output layer produces high-confidence predictions for training samples but fails to generalize to unseen validation data, causing cross-entropy loss to drop on the training set while validation accuracy stagnates.

Answer

Batch size is too large

Answer

Number of epochs is too small

Answer

Learning rate is too high

Question 4

A data scientist is using Amazon SageMaker's built-in BlazingText algorithm for word2vec embeddings. The dataset is a corpus of 10 million documents. After training, the data scientist observes that the learned embeddings do not capture semantic similarity well (e.g., 'king' and 'queen' are not close). Which hyperparameter adjustment is most likely to improve the quality of embeddings?

Accepted Answer

Increase the vector dimensionality. Increasing the vector dimensionality allows the model to capture more nuanced semantic relationships and co-occurrence patterns in the data. With 10 million documents, the default dimensionality (typically 100 or 300) may be insufficient to encode the rich contextual information, so raising it (e.g., to 300 or 500) gives the model more capacity to learn high-quality embeddings where words like 'king' and 'queen' become closer in vector space.

Answer

Decrease the window size

Answer

Decrease the number of negative samples

Answer

Increase the learning rate

Question 5

A company uses Amazon SageMaker to train a model for fraud detection. The dataset has 1 million transactions, with 0.1% fraud. The data scientist trains a random forest model and achieves 99.9% accuracy but 0% recall on the fraud class. Which technique is most likely to improve recall without significantly reducing precision?

Accepted Answer

Apply SMOTE to generate synthetic fraud samples. Option C is correct because SMOTE (Synthetic Minority Oversampling Technique) generates synthetic fraud samples by interpolating between existing minority class instances, which directly addresses the extreme class imbalance (0.1% fraud). This increases the representation of the fraud class in the training data, allowing the random forest model to learn decision boundaries that capture fraud patterns, thereby improving recall without introducing the noise or information loss associated with other methods.

Answer

Tune the classification threshold

Answer

Use cost-sensitive learning with a high cost for fraud misclassification

Answer

Undersample the majority class

Question 6

A data scientist is training a linear regression model and wants to check for multicollinearity among the features. Which TWO methods can be used to detect multicollinearity? (Choose TWO.)

Accepted Answer

Compute the correlation matrix between features. Option B is correct because computing the correlation matrix between features directly reveals pairwise linear relationships. High correlation coefficients (e.g., >0.8 or <-0.8) between two predictors indicate potential multicollinearity, which can destabilize coefficient estimates in linear regression.

Answer

Examine the R-squared value of the model

Answer

Check the p-values of the coefficients

Answer

Plot the residuals vs. fitted values

Question 7

A data scientist is training a deep learning model for object detection using Amazon SageMaker. The training job is using a single GPU instance and is taking too long. Which THREE actions can reduce training time? (Choose THREE.)

Accepted Answer

Enable mixed precision training with FP16. Option B is correct because enabling mixed precision training with FP16 reduces memory usage and accelerates computation by using half-precision floating-point numbers where possible, which is particularly effective on NVIDIA GPUs with Tensor Cores (e.g., V100, A100). This can nearly double throughput for deep learning models without sacrificing model accuracy, as critical operations still use FP32 precision.

Answer

Use a CPU instance instead of GPU

Answer

Reduce the batch size

Question 8

A data scientist is performing feature selection for a classification problem with 100 features. The data scientist wants to reduce overfitting and improve model interpretability. Which THREE methods are appropriate for feature selection? (Choose THREE.)

Accepted Answer

Recursive Feature Elimination (RFE). Recursive Feature Elimination (RFE) is a wrapper method that recursively removes the least important features based on a model's feature weights or coefficients, training the model multiple times to identify the optimal subset. This directly reduces overfitting by eliminating irrelevant or redundant features and improves interpretability by keeping only the most predictive features.

Answer

Principal Component Analysis (PCA)

Answer

Adding random noise to the features

Question 9

A data scientist ran an XGBoost training job on Amazon SageMaker using a CSV dataset. The training job failed with the error shown. What is the most likely cause of this failure?

Accepted Answer

One of the rows in the CSV has an extra column. XGBoost on SageMaker expects the CSV input to have a consistent number of columns across all rows. If one row contains an extra column, the parser will fail because it cannot map the additional field to the feature schema defined by the training job. This mismatch causes the 'Error: Number of columns does not match' or similar parsing error.

Answer

The dataset includes a header row

Answer

The delimiter used in the CSV is not a comma

Answer

The dataset contains missing values

Question 10

A data scientist is setting up an IAM role for an Amazon SageMaker training job. The policy shown is attached to the role. The training job fails with an access denied error when trying to read the training data from s3://my-bucket/training/data.csv. What is the most likely reason?

Accepted Answer

The bucket policy on my-bucket denies access to the IAM role. Option A is correct because the IAM policy shown grants s3:GetObject access to the bucket and object, but the training job still fails with an access denied error. The most likely cause is that the bucket policy on my-bucket explicitly denies access to the IAM role, overriding the IAM policy's allow. In AWS, an explicit deny in a resource-based policy (bucket policy) takes precedence over any allow in an identity-based policy (IAM role policy), causing the access denied error despite the IAM policy appearing sufficient.

Answer

The training job is using a different IAM role

Answer

The IAM policy is missing the s3:ListBucket permission

Answer

The IAM role does not have permission to access the bucket location

Question 11

A data scientist ran a hyperparameter tuning job for an XGBoost model. The tuning job completed, but the best validation RMSE is 2.34. The data scientist believes the model can perform better. Based on the exhibit, which change to the tuning strategy is most likely to improve the model's performance?

Accepted Answer

Increase the static num_round hyperparameter to 500. Option D is correct because increasing the static `num_round` hyperparameter to 500 allows the model to train for more boosting rounds, which can reduce underfitting and lower the RMSE further. The current best validation RMSE of 2.34 suggests the model may not have converged, and additional rounds can help the XGBoost model learn more complex patterns, provided overfitting is monitored with early stopping.

Answer

Use random search instead of Bayesian optimization

Answer

Change the objective to binary:logistic

Answer

Increase the maximum value of eta to 1.0

Question 12

A data scientist is training a binary classification model for fraud detection. The dataset is highly imbalanced with only 1% fraudulent transactions. The model currently achieves 99% accuracy but only catches 5% of actual fraud cases. Which metric should the data scientist focus on to better evaluate model performance?

Accepted Answer

Recall. In fraud detection with highly imbalanced data (1% fraud), accuracy is misleading because a model can achieve 99% accuracy by simply predicting 'not fraud' for all transactions. Recall (true positive rate) measures the proportion of actual fraud cases correctly identified, which is critical when the cost of missing fraud is high. The model currently catches only 5% of fraud, so improving recall is the primary goal to reduce false negatives.

Answer

Precision

Answer

Accuracy

Answer

Root Mean Squared Error (RMSE)

Question 13

A team is using SageMaker to train a deep learning model for image classification. The training job is failing with a 'CUDA out of memory' error. The team is using a p3.2xlarge instance (1 GPU, 16 GB GPU memory). The dataset consists of 256x256 RGB images. Which action is MOST likely to resolve the error without changing the instance type?

Accepted Answer

Reduce the batch size. The 'CUDA out of memory' error indicates that the GPU memory is exhausted. Reducing the batch size directly decreases the memory footprint per training step, allowing the model to fit within the 16 GB GPU memory of the p3.2xlarge instance. This is the most direct and effective fix without changing the instance type.

Answer

Increase the batch size to utilize GPU more efficiently

Answer

Enable automatic model tuning to optimize hyperparameters

Answer

Use Spot Instances to reduce cost

Question 14

A company is using SageMaker to deploy a real-time inference endpoint for a natural language processing model. The model receives input text and returns predictions. The data scientist notices that the endpoint latency increases significantly under load. Which design change would MOST effectively reduce latency?

Accepted Answer

Use an inference pipeline to combine preprocessing and model inference. Option D is correct because an inference pipeline in SageMaker allows you to chain preprocessing logic directly with the model inference within the same endpoint container. This eliminates the need for separate Lambda functions or client-side preprocessing, which reduces network round-trips and serialization overhead, thereby lowering latency under load.

Answer

Enable data capture for monitoring

Answer

Switch to batch transform for real-time predictions

Answer

Increase the number of instances behind the endpoint

Question 15

A machine learning engineer is training a linear regression model on a dataset with 50 features. After training, the model achieves high accuracy on the training set but poor accuracy on the test set. Which technique should the engineer use to address this issue?

Accepted Answer

Apply L1 or L2 regularization. The model exhibits overfitting: high training accuracy but poor test accuracy. L1 (Lasso) or L2 (Ridge) regularization penalizes large coefficients, reducing model complexity and improving generalization. This directly addresses the variance problem without requiring more data or features.

Answer

Train a deeper neural network with more layers

Answer

Add more features through feature engineering

Answer

Increase the size of the training dataset