MLS-C01 Modeling Practice Test 30 — 15 Questions

Question 1

Refer to the exhibit. A data scientist creates a SageMaker notebook instance using this Terraform configuration. The notebook fails to start. The logs indicate 'The IAM role does not have the necessary permissions'. Which addition to the IAM role policy is MOST likely needed?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

sagemaker:CreatePresignedNotebookInstanceUrl. The SageMaker notebook instance requires the `sagemaker:CreatePresignedNotebookInstanceUrl` permission to generate a presigned URL, which is used to access the notebook's Jupyter interface. Without this permission, the notebook fails to start because the IAM role cannot create the necessary URL for the user to connect, as indicated by the 'The IAM role does not have the necessary permissions' log error.

Answer

cloudwatch:PutMetricData

Answer

s3:GetObject on the notebook bucket

Answer

sagemaker:CreateTrainingJob

Question 2

A company is building a binary classification model to predict customer churn. The dataset has 10,000 samples with 500 churners (positive class). The data scientist trains a logistic regression model and obtains an accuracy of 95%. However, the model predicts all customers as non-churn. Which metric should the data scientist use to evaluate the model's performance?

Accepted Answer

F1-score. The F1-score is the harmonic mean of precision and recall, making it robust to class imbalance. Since the model predicts all customers as non-churn (accuracy 95% due to 9500 non-churners), precision for the positive class is undefined (0 true positives) and recall is 0, so the F1-score correctly reveals the model's failure to identify any churners.

Answer

AUC-ROC

Answer

Accuracy

Answer

Confusion matrix

Question 3

A data scientist is training a regression model on a dataset with 50 features. After training a linear regression model, the model achieves an R-squared of 0.85 on the training set but only 0.55 on the test set. Which technique is most likely to reduce the generalization error?

Accepted Answer

Apply L2 regularization (Ridge regression). The model exhibits high variance (overfitting): high training R² (0.85) but much lower test R² (0.55). L2 regularization (Ridge regression) shrinks coefficients toward zero, reducing model complexity and penalizing large weights, which directly combats overfitting and improves generalization to unseen data.

Answer

Add more features

Answer

Remove highly correlated features

Answer

Increase the polynomial degree of the model

Question 4

A data scientist is training a deep learning model for image classification using Amazon SageMaker. The training job is taking too long. The data scientist notices that GPU utilization is low (around 30%). Which action is most likely to improve GPU utilization and reduce training time?

Accepted Answer

Increase the batch size. Low GPU utilization (around 30%) indicates that the GPU is spending too much time idle while waiting for data batches to be processed. Increasing the batch size allows each training step to process more samples per forward/backward pass, which increases computational load on the GPU and improves hardware utilization. This directly reduces the number of steps needed per epoch, thereby decreasing overall training time.

Answer

Use a smaller instance type

Answer

Increase the learning rate

Answer

Reduce the batch size

Question 5

A company has a time series dataset of daily sales for the past 5 years. They want to forecast sales for the next 30 days. The data shows weekly seasonality and a slight upward trend. Which Amazon SageMaker algorithm is most appropriate for this task?

Accepted Answer

DeepAR. DeepAR is purpose-built for time series forecasting with seasonal patterns and trends. It uses a recurrent neural network (RNN) to model the conditional distribution of future values given past observations, and it natively handles multiple time series, missing data, and known seasonal periods (e.g., weekly). The weekly seasonality and upward trend in the daily sales data are exactly the kind of patterns DeepAR is designed to capture.

Answer

Linear Learner

Answer

XGBoost

Answer

K-Means

Question 6

A data scientist is training a binary classification model using Amazon SageMaker's XGBoost. The dataset is highly imbalanced (99% negative class, 1% positive class). The data scientist wants to maximize the F1-score. Which parameter adjustment is most appropriate?

Accepted Answer

Set scale_pos_weight to 99. Setting scale_pos_weight to 99 is the most appropriate adjustment because it directly addresses class imbalance by assigning a higher weight to the minority (positive) class during training. In XGBoost, scale_pos_weight controls the balance of positive and negative weights, typically set as sum(negative instances) / sum(positive instances), which here is 99/1 = 99. This forces the model to penalize misclassifications of the positive class more heavily, thereby improving recall and F1-score.

Answer

Set max_depth to 10

Answer

Set eta to 0.01

Answer

Set subsample to 0.5

Question 7

A data scientist is using Amazon SageMaker to train a model on a dataset that contains both numerical and categorical features. The categorical features have high cardinality (e.g., postal codes, product IDs). Which feature engineering approach is most suitable for handling these high-cardinality categorical features in a tree-based model?

Accepted Answer

Use label encoding. Label encoding is suitable for tree-based models because these models split on feature values and can handle ordinal relationships implicitly. Unlike linear models, tree-based models do not assume any distance metric between categories, so label encoding avoids the dimensionality explosion of one-hot encoding while preserving the ability to capture splits based on high-cardinality features.

Answer

One-hot encode the categorical features

Answer

Apply target encoding

Answer

Apply binary encoding

Question 8

A data scientist is training a neural network for a multi-class classification problem with 100 classes. The model uses a softmax output layer and cross-entropy loss. During training, the loss decreases steadily but the accuracy on the validation set plateaus early. Which of the following is the most likely cause?

Accepted Answer

The model is overfitting the training data. When the validation accuracy plateaus early while training loss continues to decrease, it indicates that the model is memorizing the training data rather than learning generalizable patterns. This is classic overfitting, where the softmax output layer produces high-confidence predictions for training samples but fails to generalize to unseen validation data, causing cross-entropy loss to drop on the training set while validation accuracy stagnates.

Answer

Batch size is too large

Answer

Number of epochs is too small

Answer

Learning rate is too high

Question 9

A data scientist is using Amazon SageMaker's built-in BlazingText algorithm for word2vec embeddings. The dataset is a corpus of 10 million documents. After training, the data scientist observes that the learned embeddings do not capture semantic similarity well (e.g., 'king' and 'queen' are not close). Which hyperparameter adjustment is most likely to improve the quality of embeddings?

Accepted Answer

Increase the vector dimensionality. Increasing the vector dimensionality allows the model to capture more nuanced semantic relationships and co-occurrence patterns in the data. With 10 million documents, the default dimensionality (typically 100 or 300) may be insufficient to encode the rich contextual information, so raising it (e.g., to 300 or 500) gives the model more capacity to learn high-quality embeddings where words like 'king' and 'queen' become closer in vector space.

Answer

Decrease the window size

Answer

Decrease the number of negative samples

Answer

Increase the learning rate

Question 10

A company uses Amazon SageMaker to train a model for fraud detection. The dataset has 1 million transactions, with 0.1% fraud. The data scientist trains a random forest model and achieves 99.9% accuracy but 0% recall on the fraud class. Which technique is most likely to improve recall without significantly reducing precision?

Accepted Answer

Apply SMOTE to generate synthetic fraud samples. SMOTE (Synthetic Minority Oversampling Technique) generates synthetic fraud samples by interpolating between existing minority class instances, which directly addresses the extreme class imbalance (0.1% fraud). This increases the representation of the fraud class in the training data, allowing the random forest model to learn decision boundaries that capture fraud patterns, thereby improving recall without introducing the noise or information loss associated with other methods.

Answer

Tune the classification threshold

Answer

Use cost-sensitive learning with a high cost for fraud misclassification

Answer

Undersample the majority class

Question 11

A data scientist is training a linear regression model and wants to check for multicollinearity among the features. Which TWO methods can be used to detect multicollinearity? (Choose TWO.)

Accepted Answer

Compute the correlation matrix between features. Computing the correlation matrix between features directly reveals pairwise linear relationships. High correlation coefficients (e.g., >0.8 or <-0.8) between two predictors indicate potential multicollinearity, which can destabilize coefficient estimates in linear regression.

Answer

Examine the R-squared value of the model

Answer

Check the p-values of the coefficients

Answer

Plot the residuals vs. fitted values

Question 12

A data scientist is training a deep learning model for object detection using Amazon SageMaker. The training job is using a single GPU instance and is taking too long. Which THREE actions can reduce training time? (Choose THREE.)

Accepted Answer

Enable mixed precision training with FP16. Enabling mixed precision training with FP16 reduces memory usage and accelerates computation by using half-precision floating-point numbers where possible, which is particularly effective on NVIDIA GPUs with Tensor Cores (e.g., V100, A100). This can nearly double throughput for deep learning models without sacrificing model accuracy, as critical operations still use FP32 precision.

Answer

Use a CPU instance instead of GPU

Answer

Reduce the batch size

Question 13

A data scientist is performing feature selection for a classification problem with 100 features. The data scientist wants to reduce overfitting and improve model interpretability. Which THREE methods are appropriate for feature selection? (Choose THREE.)

Accepted Answer

Recursive Feature Elimination (RFE). Recursive Feature Elimination (RFE) is a wrapper method that recursively removes the least important features based on a model's feature weights or coefficients, training the model multiple times to identify the optimal subset. This directly reduces overfitting by eliminating irrelevant or redundant features and improves interpretability by keeping only the most predictive features.

Answer

Principal Component Analysis (PCA)

Answer

Adding random noise to the features

Question 14

A data scientist ran an XGBoost training job on Amazon SageMaker using a CSV dataset. The training job failed with the error shown. What is the most likely cause of this failure?

Accepted Answer

One of the rows in the CSV has an extra column. XGBoost on SageMaker expects the CSV input to have a consistent number of columns across all rows. If one row contains an extra column, the parser will fail because it cannot map the additional field to the feature schema defined by the training job. This mismatch causes the 'Error: Number of columns does not match' or similar parsing error.

Answer

The dataset includes a header row

Answer

The delimiter used in the CSV is not a comma

Answer

The dataset contains missing values

Question 15

A data scientist is setting up an IAM role for an Amazon SageMaker training job. The policy shown is attached to the role. The training job fails with an access denied error when trying to read the training data from s3://my-bucket/training/data.csv. What is the most likely reason?

Accepted Answer

The bucket policy on my-bucket denies access to the IAM role. The IAM policy shown grants s3:GetObject access to the bucket and object, but the training job still fails with an access denied error. The most likely cause is that the bucket policy on my-bucket explicitly denies access to the IAM role, overriding the IAM policy's allow. In AWS, an explicit deny in a resource-based policy (bucket policy) takes precedence over any allow in an identity-based policy (IAM role policy), causing the access denied error despite the IAM policy appearing sufficient.

Answer

The training job is using a different IAM role

Answer

The IAM policy is missing the s3:ListBucket permission

Answer

The IAM role does not have permission to access the bucket location