MLS-C01 Practice Test 19 — 15 Questions

Question 1

A team trained a multiclass classification model using SageMaker built-in XGBoost. The model's accuracy is high, but for a specific class, recall is very low. The team wants to improve recall for that class without significant accuracy drop. Which approach is MOST effective?

Accepted Answer

Resample the training data to balance the class representation. Option B is correct because resampling the training data to balance class representation directly addresses the root cause of low recall for a specific class in a multiclass XGBoost model. XGBoost's built-in objective functions (e.g., 'multi:softmax') optimize for overall accuracy, which can bias the model toward majority classes; resampling (e.g., oversampling the minority class or undersampling the majority) forces the model to learn decision boundaries that better capture the minority class, improving recall without drastically reducing overall accuracy.

Answer

Add more training data from all classes

Answer

Increase the max_depth hyperparameter of XGBoost

Answer

Switch from XGBoost to a linear learner

Question 2

A machine learning engineer is evaluating a binary classification model. The model has a high recall but low precision. Which of the following is the most likely consequence?

Accepted Answer

The model has many false positives.. High recall means the model correctly identifies most positive cases (few false negatives), but low precision indicates that among the cases predicted as positive, many are actually negative. This directly implies a high number of false positives, as precision = TP/(TP+FP) and a low precision with high recall forces FP to be large relative to TP.

Answer

The model has few false negatives.

Answer

The model misses many positive cases.

Answer

The model has few false positives.

Question 3

A data scientist is training a binary classifier on a dataset with 1 million rows and 500 features. The model uses XGBoost and achieves an AUC of 0.95 on the training set but only 0.72 on the test set. The scientist suspects overfitting. Which combination of hyperparameter adjustments is most likely to improve generalization?

Accepted Answer

Decrease 'max_depth' and increase 'min_child_weight'. Option C is correct because decreasing 'max_depth' reduces the complexity of individual trees, preventing the model from learning overly specific patterns in the training data. Increasing 'min_child_child_weight' forces the algorithm to require a higher sum of instance weights (hessian) before further partitioning, which acts as a regularization mechanism that discourages splits on noisy or sparse data. Together, these adjustments directly combat overfitting in XGBoost by limiting tree depth and requiring more evidence for splits, which improves generalization from the training AUC of 0.95 to a higher test AUC.

Answer

Increase 'max_depth' and decrease 'learning_rate'

Answer

Increase 'subsample' and decrease 'colsample_bytree'

Answer

Decrease 'gamma' and increase 'learning_rate'

Question 4

A data scientist is evaluating a classification model. The confusion matrix shows that the model has 50 true positives, 100 true negatives, 20 false positives, and 30 false negatives. Which TWO metrics can be calculated from this confusion matrix? (Choose two.)

Accepted Answer

Recall. Options A and D are correct because precision and recall are directly computed from TP, FP, FN. Option B is wrong because R-squared is for regression. Option C is wrong because RMSE is for regression. Option E is wrong because F1 score requires both precision and recall, but it is not directly from the confusion matrix without calculation; however, it can be derived. But the question asks 'calculate from this confusion matrix', and both precision and recall are directly calculated. F1 is derived from them, so it's also calculable. However, the question says 'TWO', and the most direct are precision and recall. F1 requires an extra step. I'll consider precision and recall as the correct pair. But to be precise, I'll pick A and D.

Answer

R-squared

Answer

F1 score

Answer

Root mean squared error

Question 5

A data scientist is building a recommendation system for an e-commerce platform. The dataset includes user-item interactions (clicks, purchases, ratings). The scientist wants to use matrix factorization. Which approach is most appropriate for handling implicit feedback (e.g., clicks) rather than explicit ratings?

Accepted Answer

Use weighted alternating least squares (WALS) with confidence weights. Weighted Alternating Least Squares (WALS) is specifically designed for implicit feedback scenarios because it treats unobserved interactions as negative signals with low confidence, rather than missing values. By assigning confidence weights (e.g., based on click frequency or dwell time), WALS can factorize the implicit feedback matrix effectively, avoiding the bias introduced by treating all zeros as true negatives.

Answer

Use k-means clustering to segment users and then use item popularity within clusters

Answer

Use singular value decomposition (SVD) on the interaction matrix with missing values filled with 0

Answer

Use a deep neural network with a softmax output to predict item probabilities

Question 6

A company runs an e-commerce platform on AWS. They have a SageMaker endpoint serving a product recommendation model. The model uses a custom container with a TensorFlow model. Recently, the endpoint has been returning high latency and occasional 504 errors during peak traffic. The data scientist observes that the model inference time is around 200 ms per request, but the endpoint is configured with a single ml.c5.large instance. The traffic spikes can reach 100 requests per second. The data scientist needs to reduce latency and eliminate 504 errors. Which course of action is most appropriate?

Accepted Answer

Configure the SageMaker endpoint with Application Auto Scaling to scale out based on the 'InvocationsPerInstance' metric, and use a larger instance type such as ml.c5.xlarge. Option B is correct because the endpoint is bottlenecked by both instance size and concurrency. With a single ml.c5.large instance handling 100 requests per second and a 200 ms inference time, the instance can only process about 5 requests per second (1000 ms / 200 ms = 5 requests per second per instance). Application Auto Scaling based on the 'InvocationsPerInstance' metric will add instances during traffic spikes, while upgrading to ml.c5.xlarge doubles compute capacity per instance, reducing latency and eliminating 504 errors caused by request queue overflow.

Answer

Use Amazon Elastic Inference to attach an EI accelerator to the endpoint instance

Answer

Switch to a multi-model endpoint to serve multiple models on the same instance

Answer

Replace the SageMaker endpoint with an AWS Lambda function that loads the model from S3 and returns predictions

Question 7

A team is deploying a SageMaker endpoint for a model that was trained with scikit-learn. The endpoint receives spikes in traffic during business hours. The team wants to minimize cost while ensuring availability during spikes. Which endpoint configuration is MOST appropriate?

Accepted Answer

Use a production variant endpoint with auto-scaling based on CPU utilization. Option B is correct because a production variant endpoint with auto-scaling based on CPU utilization allows the SageMaker endpoint to dynamically adjust the number of instances in response to traffic spikes, ensuring availability during business hours while minimizing cost by scaling down during off-peak periods. This approach is ideal for a scikit-learn model, which is CPU-bound, making CPU utilization a relevant and effective scaling metric.

Answer

Use SageMaker Serverless Inference

Answer

Use a multi-model endpoint with a single instance type

Answer

Deploy a single large instance that can handle peak load

Question 8

A data scientist is using Amazon SageMaker to deploy a model for real-time inference. The model is a TensorFlow neural network. The scientist wants to use automatic scaling based on the number of incoming requests. Which service integration is required?

Accepted Answer

Amazon SageMaker endpoint configured with Application Auto Scaling. Amazon SageMaker endpoints natively integrate with Application Auto Scaling to adjust the number of instances based on a target metric, such as the number of incoming requests per instance. This allows the TensorFlow model to scale automatically in response to traffic, without needing additional orchestration services.

Answer

Amazon ECS with service auto scaling

Answer

AWS Lambda with provisioned concurrency

Answer

AWS Auto Scaling plans

Question 9

A data scientist is training a binary classification model on a dataset with 100 features and 10,000 rows. The model overfits significantly: training accuracy is 99%, but validation accuracy is 80%. The data scientist has tried L1 and L2 regularization without improvement. The dataset is clean and representative. Which approach is MOST likely to reduce overfitting? A. Increase the number of training epochs. B. Add more training data by generating synthetic samples using SMOTE. C. Reduce the number of features using PCA. D. Use a simpler model like logistic regression instead of a decision tree ensemble. The data scientist needs to maintain a validation accuracy above 85%, but the current model is too complex. The company has limited budget for data labeling. Which option is BEST?

Accepted Answer

Use a simpler model like logistic regression. Option A is correct because the current model (likely a decision tree ensemble like Random Forest or XGBoost) is too complex for the dataset, causing overfitting. Switching to a simpler model like logistic regression reduces variance by limiting the hypothesis space, which directly addresses overfitting without requiring additional data or feature engineering. Given the limited labeling budget, this approach is cost-effective and can improve generalization, potentially achieving the required >85% validation accuracy.

Answer

Add more training data by generating synthetic samples using SMOTE

Answer

Reduce the number of features using PCA

Answer

Increase the number of training epochs

Question 10

A company uses SageMaker to train a model that processes sensitive customer data. Due to compliance, the training data must be encrypted at rest and in transit, and the model artifacts must be stored in a secured S3 bucket with encryption. Which combination of actions is REQUIRED?

Accepted Answer

Enable AWS KMS encryption for the SageMaker notebook and training job, and use an S3 bucket with default encryption using AWS KMS. Option D is correct because it ensures end-to-end encryption: AWS KMS encryption for the SageMaker notebook and training job encrypts data in transit and at rest within the SageMaker environment, while an S3 bucket with default encryption using AWS KMS encrypts the training data and model artifacts at rest. This combination meets compliance requirements for encryption at rest and in transit, as KMS provides envelope encryption with customer-managed keys, and SageMaker automatically uses TLS for data in transit when KMS is enabled.

Answer

Store data in an S3 bucket with AWS CloudHSM integration

Answer

Use an S3 bucket with SSE-S3 and enable SageMaker Internet-facing mode

Answer

Use an S3 bucket with default encryption (SSE-S3) and enable SSL for all connections

Question 11

A data scientist is using Amazon SageMaker to train a linear regression model. The training data contains missing values. Which TWO techniques are appropriate for handling missing values in the dataset?

Accepted Answer

Remove rows with missing values if the proportion is small. Options A (Impute with the mean of the column) and C (Remove rows with missing values if the proportion is small) are correct. Imputation is a common technique; removing rows is acceptable if few missing. Option B (Set missing values to zero) can bias the model. Option D (Use a decision tree that handles missing values internally) is not applicable for linear regression. Option E (Use a separate category for missing) is not suitable for linear regression numeric features.

Answer

Use a decision tree model that can handle missing values internally

Answer

Set missing values to zero

Answer

Create a separate category for missing values

Question 12

Which TWO actions should a data scientist take when exploring a dataset that contains missing values and outliers? (Select TWO.)

Accepted Answer

Calculate the percentage of missing values per column.. Options B and E are correct. B: Visualizing the distribution helps identify shape and outliers. E: Reporting missing value counts is a standard EDA step. A is wrong because imputation should be done after analysis. C is wrong because removing outliers without analysis is premature. D is wrong because normalization is not a first step.

Answer

Normalize all features using Min-Max scaling.

Answer

Remove all rows with outliers.

Answer

Impute missing values with the mean immediately.

Question 13

A team is training a deep learning model on SageMaker using a custom PyTorch container. Training takes 24 hours on a single ml.p3.2xlarge instance. The team wants to reduce training time using distributed training. Which strategy is MOST appropriate?

Accepted Answer

Use model parallelism to split the model across multiple GPUs. Option C (Model parallelism) is correct for deep learning models that are too large to fit on a single GPU. Option A (Data parallelism) would require the model to fit on each GPU, which may not be the case. Option B (Hyperparameter tuning) does not reduce training time directly. Option D (Spot instances) may cause interruptions and does not guarantee speedup.

Answer

Use data parallelism with Horovod across multiple instances

Answer

Use SageMaker Managed Spot Training to reduce cost

Answer

Use SageMaker Automatic Model Tuning to find optimal hyperparameters

Question 14

A data scientist ran a SageMaker training job that failed with the error shown. The training script expects the data in '/opt/ml/input/data/training/train.csv'. What is the most likely issue?

Accepted Answer

The S3 data path should point to the exact file, not the folder. Option A is correct. The S3Uri points to a prefix (folder), but the script expects a specific file name. SageMaker copies the contents of the S3 prefix to the channel directory; if the file is not directly under that prefix, it won't be found. Option B is wrong because the channel name matches. Option C is wrong if the file exists. Option D is wrong because the error is about the file, not hyperparameters.

Answer

The hyperparameter 'sagemaker_program' is misspelled

Answer

The training script has a bug in reading the file

Answer

The channel name should be 'train' instead of 'training'

Question 15

A data scientist is training a random forest model for regression. The model shows high variance on the validation set. Which TWO actions are most likely to reduce variance? (Choose 2.)

Accepted Answer

Decrease the maximum depth of trees. Decreasing depth and increasing min samples per leaf both reduce complexity, lowering variance.

Answer

Use bootstrap sampling with replacement

Answer

Increase the number of trees in the forest

Answer

Increase the number of features considered at each split