MLS-C01 Mock Exam 75 — 25 Questions

Question 1

A machine learning team is building a model to predict customer churn. They have a dataset with 10,000 samples and 50 features, including categorical variables with high cardinality (e.g., ZIP code). Which feature engineering technique is most appropriate to reduce dimensionality while preserving predictive information?

Accepted Answer

Target encoding. Target encoding replaces high-cardinality categories with the mean target value, reducing dimensionality while capturing predictive signal. Option A is wrong because one-hot encoding creates many sparse features, increasing dimensionality. Option B is wrong because PCA is applied to numerical features, not categorical. Option D is wrong because label encoding imposes ordinality that may not exist.

Answer

Principal Component Analysis (PCA)

Answer

One-hot encoding

Answer

Label encoding

Question 2

A company is using Amazon SageMaker to deploy a model that predicts customer churn. The model was trained using a linear learner algorithm. During inference, the endpoint returns predictions that are always 0.5 (the probability of churn). What is the most likely cause?

Accepted Answer

The learning rate is set too high, causing the model to converge to the mean prediction. If the model always outputs 0.5, it suggests that the model is not learning and is stuck at the prior probability. This often happens when the learning rate is too high (causing divergence) or too low (causing slow convergence) so that the model does not update weights. The other options would cause different symptoms: data imbalance might bias towards 0 or 1, not exactly 0.5; feature scaling issues typically cause NaN or poor convergence; insufficient epochs might not converge but not necessarily give exactly 0.5.

Answer

The dataset is highly imbalanced, and the model is predicting the majority class

Answer

The model was trained with too few epochs

Answer

The input features are not normalized

Question 3

A data scientist is training a deep learning model on a large dataset using Amazon SageMaker. The training job is taking too long. The scientist notices that GPU utilization is low and data loading is the bottleneck. Which action should the scientist take to improve training performance?

Accepted Answer

Use Pipe mode for the training data channel. Low GPU utilization with data loading bottleneck indicates that the CPU cannot feed data fast enough. Using Pipe mode streams data directly from S3 without downloading, reducing I/O overhead. Increasing instance count may not help if each GPU is underutilized. Changing to a CPU instance would be slower. Reducing batch size would reduce GPU utilization further. Option A: Pipe mode is correct. Option B: More instances may not address data loading bottleneck. Option C: CPU instance is slower. Option D: Smaller batch size reduces GPU utilization.

Answer

Increase the number of training instances

Answer

Change the instance type to a CPU instance

Answer

Reduce the batch size

Question 4

A data scientist is training a neural network on a dataset with 1 million images. The training loss decreases steadily but the validation loss starts to increase after 10 epochs. Which action should the scientist take to improve generalization?

Accepted Answer

Implement early stopping. Increasing validation loss indicates overfitting. Early stopping halts training when validation loss stops improving, preventing overfitting. Increasing epochs would worsen overfitting. Reducing learning rate might help but early stopping directly addresses the issue. Adding more layers could increase overfitting. Option A: Early stopping is correct. Option B: Increasing epochs would worsen overfitting. Option C: Reducing learning rate might help but not as directly. Option D: Adding more layers could increase overfitting.

Answer

Add more layers to the network

Answer

Reduce the learning rate

Answer

Increase the number of epochs

Question 5

A data scientist is training a k-means clustering model on a dataset with 1,000 points. The scientist uses the elbow method to choose the number of clusters. The elbow plot shows a clear bend at k=4. After running k-means with k=4, the scientist wants to evaluate the quality of the clustering. Which THREE of the following are suitable internal clustering validation metrics? (Choose THREE.)

Accepted Answer

Calinski-Harabasz index. Silhouette score, Davies-Bouldin index, and Calinski-Harabasz index are all internal validation metrics that do not require ground truth labels. They measure compactness and separation. Rand index and adjusted Rand index require ground truth labels (external validation).

Answer

Adjusted Rand index

Answer

Rand index

Question 6

A company is building a multiclass classification model using Amazon SageMaker. The dataset has 100 classes and is highly imbalanced. The model currently achieves high accuracy on the majority classes but poor performance on minority classes. Which technique should the data scientist use to improve minority class performance?

Accepted Answer

Use class weights to penalize misclassifications of minority classes. Option C is correct because class weights penalize errors on minority classes more, improving their recall. Option A is wrong because removing majority classes would lose data. Option B is wrong because oversampling without replacement may cause overfitting. Option D is wrong because principal component analysis (PCA) is dimensionality reduction, not for imbalance.

Answer

Apply random oversampling with replacement

Answer

Apply principal component analysis (PCA)

Answer

Remove samples from majority classes

Question 7

A data scientist is working on a multi-class classification problem with 10 classes. The model outputs probabilities and the scientist wants to evaluate the model's ability to rank classes correctly. Which metric is most appropriate?

Accepted Answer

Area Under the ROC Curve (AUC-ROC). The Area Under the Receiver Operating Characteristic Curve (AUC-ROC) measures the model's ability to distinguish between classes. For multi-class, one-vs-rest AUC can be used. Log loss measures calibration, not ranking. F1 score is for binary classification or per-class. Accuracy does not consider ranking. Option A: AUC-ROC is correct. Option B: Log loss measures probability calibration. Option C: F1 score is not a ranking metric. Option D: Accuracy is not a ranking metric.

Answer

F1 score

Answer

Accuracy

Answer

Log loss

Question 8

Which THREE techniques are effective for reducing overfitting in a deep neural network?

Accepted Answer

Early stopping. Dropout randomly drops neurons during training, L2 regularization penalizes large weights, and early stopping halts training before overfitting. Increasing model complexity (more layers) would increase overfitting. Data augmentation is also effective but not listed as a separate option; here we have dropout, L2, and early stopping. So correct are A, B, D. C: Increasing model complexity is opposite. E: Reducing training data would worsen overfitting.

Answer

Increasing model complexity

Answer

Reducing the amount of training data

Question 9

A data scientist is training a linear regression model on a dataset with 10 numerical features. After training, the model's R-squared value is 0.99 on the training set but only 0.60 on the test set. Which TWO of the following are appropriate actions to reduce overfitting? (Choose TWO.)

Accepted Answer

Use a subset of the most important features. Regularization (L1 or L2) penalizes large coefficients and reduces overfitting. Reducing model complexity by using fewer features or simplifying the model also helps. Adding more features would increase complexity and overfitting. Increasing the number of epochs is not relevant for linear regression (which has a closed-form solution).

Answer

Normalize the features

Answer

Add more features to the model

Answer

Increase the number of training epochs

Question 10

A machine learning engineer is deploying a model for real-time inference using Amazon SageMaker. The model is a large ensemble that requires 8 GB of memory and 4 vCPUs. The expected traffic is 100 requests per second with a 200 ms latency requirement. Which instance configuration should they choose?

Accepted Answer

ml.c5.2xlarge (8 vCPU, 16 GB). Option C is correct because ml.c5.2xlarge has 8 vCPUs and 16 GB memory, which is sufficient and cost-effective for the workload. Option A is wrong because ml.m5.large has only 2 vCPUs and 8 GB memory, which may not handle the throughput. Option B is wrong because ml.p3.2xlarge is GPU-optimized and overkill. Option D is wrong because ml.t2.medium is too small for the memory requirement.

Answer

ml.t2.medium (2 vCPU, 4 GB)

Answer

ml.p3.2xlarge (8 vCPU, 61 GB GPU)

Answer

ml.m5.large (2 vCPU, 8 GB)

Question 11

A data scientist is training a random forest model on a dataset with 50 features. After training, the model achieves 98% accuracy on the training set but only 85% on the test set. Which technique is most appropriate to reduce the generalization error?

Accepted Answer

Reduce the maximum depth of each tree. The gap indicates overfitting. Random forest can overfit if trees are too deep or if the number of trees is too high. Reducing the maximum depth of trees limits model complexity and helps generalization. Increasing the number of trees typically reduces overfitting but can also increase computational cost; however, reducing depth is more direct. Feature selection or PCA might help but are less direct than controlling tree complexity.

Answer

Apply Principal Component Analysis (PCA) to reduce dimensionality

Answer

Add more training data

Answer

Increase the number of trees in the forest

Question 12

A data scientist is using SageMaker to train a random forest model. The dataset has 100 features and 1 million rows. The training job fails with a 'ResourceLimitExceeded' error. What is the MOST likely cause?

Accepted Answer

The account has reached its limit on the number of SageMaker training instances.. Random forest usually does not require GPU; CPU instances are sufficient. The error is likely due to exceeding the allowed number of instances or vCPU limit. Option A (GPU memory) is unlikely as random forest uses CPU. Option C (S3 bucket) does not cause resource limit. Option D (wrong algorithm) would give different error.

Answer

The S3 bucket containing the training data is not in the same region.

Answer

The instance type selected does not have enough GPU memory.

Answer

The wrong algorithm was specified for the training job.

Question 13

A data scientist is training a binary classification model on a highly imbalanced dataset where the positive class represents only 1% of the data. Which metric should be used to evaluate model performance during training to ensure the model is learning to detect the positive class?

Accepted Answer

F1 score. Accuracy is misleading for imbalanced datasets because a model that predicts the majority class all the time can achieve 99% accuracy. F1 score balances precision and recall, making it suitable for imbalanced classification. Precision, recall, and AUC are also useful, but F1 is a common single metric for imbalanced binary classification. Option A: Accuracy is not suitable. Option B: Precision alone ignores recall. Option C: F1 score is correct. Option D: Recall alone ignores precision.

Answer

Accuracy

Answer

Precision

Answer

Recall

Question 14

A machine learning team is deploying a model that performs real-time inference on streaming data from Amazon Kinesis Data Streams. The model requires sub-100ms latency. Which deployment option should the team choose?

Accepted Answer

Deploy the model on an Amazon SageMaker real-time endpoint. Amazon SageMaker real-time endpoints provide low-latency inference suitable for sub-100ms requirements. SageMaker batch transform is for offline predictions. SageMaker asynchronous inference is for near-real-time with longer latencies. AWS Lambda alone may not handle model serving efficiently for low latency. Option A: SageMaker real-time endpoint is correct. Option B: SageMaker batch transform is for batch, not real-time. Option C: SageMaker asynchronous inference has higher latency. Option D: AWS Lambda custom inference is possible but may not meet sub-100ms consistently due to cold starts.

Answer

Use Amazon SageMaker batch transform

Answer

Use Amazon SageMaker asynchronous inference

Answer

Deploy a custom inference container on AWS Lambda

Question 15

A machine learning engineer is deploying a model using Amazon SageMaker and wants to automatically scale the endpoint based on the number of incoming requests. Which scaling policy should be used?

Accepted Answer

Target tracking scaling. SageMaker endpoints support Application Auto Scaling, which can use a target tracking scaling policy based on a metric like InvocationsPerInstance. Simple scaling and step scaling are also possible but target tracking is simpler. Scheduled scaling is for predictable traffic. Option A: Target tracking scaling is correct. Option B: Simple scaling requires manual thresholds. Option C: Step scaling is more complex. Option D: Scheduled scaling is for predictable patterns.

Answer

Step scaling

Answer

Scheduled scaling

Answer

Simple scaling

Question 16

Which TWO actions are best practices for tuning hyperparameters using Amazon SageMaker Automatic Model Tuning?

Accepted Answer

Use Bayesian optimization strategy. Using random search or Bayesian optimization are supported strategies. Grid search is also possible but not efficient for many hyperparameters. Setting a large number of training jobs can be costly. Using the same hyperparameters as the baseline does not tune. So correct are A and C. B: Grid search is less efficient. D: Large number of jobs is not a best practice due to cost. E: Not tuning is not a best practice.

Answer

Set the number of training jobs to a very large value

Answer

Use the same hyperparameters as the baseline model

Answer

Use grid search strategy

Question 17

A data scientist is training a binary classifier using logistic regression on a dataset that is highly imbalanced (95% negative class, 5% positive class). The model achieves 95% accuracy but only predicts the negative class. Which metric should the scientist use to evaluate the model's performance on the positive class?

Accepted Answer

Precision. Precision measures the proportion of positive identifications that were actually correct, which is suitable for imbalanced datasets. Accuracy is misleading because the model predicts only the majority class. Recall would be high if the model predicted all positives, but here it predicts none. F1 score is a harmonic mean of precision and recall, but precision is more directly relevant to the issue of false positives.

Answer

Recall

Answer

F1 Score

Answer

Accuracy

Question 18

A company uses Amazon SageMaker to build a text classification model using a pre-trained BERT model. The dataset contains 10,000 labeled documents. The model is overfitting: training accuracy is 99%, validation accuracy is 85%. Which TWO of the following are most likely to help reduce overfitting? (Choose TWO.)

Accepted Answer

Increase the dropout rate during fine-tuning. Increasing dropout during fine-tuning adds regularization. Decreasing the learning rate can help the model converge to a better solution and prevent overfitting to the training set. Increasing batch size can sometimes regularize but is not as effective as dropout. Adding more layers increases model capacity and overfitting. Using a larger pre-trained model also increases capacity.

Answer

Add more transformer layers to the model

Answer

Increase the batch size

Answer

Use a larger pre-trained BERT model

Question 19

A data scientist is using Amazon SageMaker to train a linear regression model. The training data has 10 features, and the scientist wants to interpret the model's coefficients. Which algorithm should they use?

Accepted Answer

Amazon SageMaker Linear Learner. Option A is correct because Linear Learner provides interpretable coefficients. Option B is wrong because XGBoost is tree-based and less interpretable. Option C is wrong because K-Means is unsupervised. Option D is wrong because Factorization Machines are for high-dimensional sparse data.

Answer

Amazon SageMaker XGBoost

Answer

Amazon SageMaker K-Means

Answer

Amazon SageMaker Factorization Machines

Question 20

A company uses Amazon SageMaker to train a model using the built-in Linear Learner algorithm. The training data contains missing values in some features. What is the best practice for handling missing values with this algorithm?

Accepted Answer

Impute missing values using mean or median imputation. Linear Learner expects dense input; it cannot handle missing values. The best practice is to impute missing values before training, such as using mean or median imputation. Setting the missing value to zero could bias the model. Removing rows with missing values may lose data. The algorithm does not have a built-in missing value handling parameter. Option A: Impute missing values is correct. Option B: Zero may cause bias. Option C: Removing rows may lose data. Option D: There is no parameter for missing values in Linear Learner.

Answer

Remove rows with missing values

Answer

Set missing values to zero

Answer

Use the `handle_missing` parameter in the algorithm