MLS-C01 Modeling Practice Test 3 — 15 Questions

Question 1

A data scientist is training a model using Amazon SageMaker. The training job is running on GPU instances, but the GPU utilization is low. Which TWO actions could improve GPU utilization?

Accepted Answer

Use a larger instance with multiple GPUs. Option B is correct because moving to a larger instance with multiple GPUs (e.g., an ml.p3.16xlarge or ml.p4d.24xlarge) allows distributed training across more devices, so the workload can be parallelized and each GPU receives enough work to stay busy rather than idling. Option C is correct because increasing the batch size gives each GPU more samples to process per step, raising the compute-to-overhead ratio and keeping the GPU cores saturated instead of waiting on data loading or kernel launch overhead. Options A, D, and E are not appropriate: increasing epochs only extends training time without changing per-step GPU efficiency, switching to CPU instances removes GPU acceleration entirely, and decreasing the batch size reduces the work per step, which typically worsens GPU underutilization.

Answer

Increase the number of epochs

Answer

Switch to CPU instances

Answer

Decrease the batch size

Question 2

A data scientist is performing feature engineering for a machine learning model. The dataset contains categorical features with high cardinality. Which THREE techniques are appropriate for encoding high-cardinality categorical features?

Accepted Answer

Target encoding. Target encoding (A) is appropriate because it replaces each high-cardinality category with a statistic (typically the mean) of the target variable, collapsing thousands of levels into a single numeric column while capturing predictive signal. Count encoding (D) is appropriate because it maps each category to its frequency in the dataset, producing one compact numeric feature that preserves information about rare versus common levels without exploding dimensionality. One-hot encoding with pruning of rare categories (E) is appropriate because it first groups infrequent levels (e.g., those below a minimum frequency threshold) into an 'other' bucket before creating binary columns, keeping the feature space manageable while retaining the most informative categories. Binary encoding (B) is not marked correct here, and label encoding (C) is not marked correct because it imposes an arbitrary ordinal relationship on nominal categories, which can mislead models that interpret numeric magnitude.

Answer

Binary encoding

Answer

Label encoding

Question 3

Refer to the exhibit. A training job failed with the error shown. What is the most likely cause?

Accepted Answer

The training data contains missing values or outliers that cause numerical instability. The most likely cause is that the training data contains missing values or outliers that cause numerical instability. This often manifests as errors like 'NaN' or 'Inf' during training, leading to job failure. Missing values can propagate through computations, and outliers can cause gradients to explode, resulting in numerical instability.

Answer

The model architecture is incorrect

Answer

The instance type does not have enough memory

Answer

The training job exceeded the maximum runtime

Question 4

Refer to the exhibit. A data scientist is trying to create a SageMaker training job but receives an access denied error. The IAM policy shown is attached to the user. What is the likely issue?

Accepted Answer

The policy is missing s3:ListBucket permission. SageMaker training jobs read training data from S3, and the IAM policy must grant s3:ListBucket on the bucket in addition to object-level permissions like s3:GetObject. If s3:ListBucket is missing, the training job's data download step fails with an access denied error even though the user can read individual objects. The exhibit's policy is missing that bucket-level permission, making it the likely cause.

Answer

The policy does not allow sagemaker:CreateTrainingJob

Answer

The policy does not allow sagemaker:CreateModel

Answer

The policy does not allow s3:PutObject

Question 5

Refer to the exhibit. A data scientist checks the status of a SageMaker endpoint and sees the output above. What does this indicate?

Accepted Answer

The endpoint is scaling up to meet desired capacity. When a SageMaker endpoint shows a status indicating it is provisioning additional instances to reach the desired instance count, it is scaling up. The exhibit typically shows 'Updating' or 'Creating' with a target instance count higher than the current in-service count. This is normal autoscaling behavior, not a failure.

Answer

The endpoint has failed

Answer

The endpoint is running at full capacity

Answer

The endpoint is out of service

Question 6

A machine learning team is building a model to predict customer churn. They have a dataset with 10,000 samples and 50 features, including categorical variables with high cardinality (e.g., ZIP code). Which feature engineering technique is most appropriate to reduce dimensionality while preserving predictive information?

Accepted Answer

Target encoding. Target encoding replaces high-cardinality categories with the mean target value, reducing dimensionality while capturing predictive signal. Option A (PCA) is wrong because PCA is applied to numerical features, not categorical. Option B (One-hot encoding) is wrong because one-hot encoding creates many sparse features, increasing dimensionality. Option D is wrong because label encoding imposes ordinality that may not exist.

Answer

Principal Component Analysis (PCA)

Answer

One-hot encoding

Answer

Label encoding

Question 7

A machine learning engineer is using Amazon SageMaker to deploy a model for real-time inference. The model is a large ensemble that requires 4 GB of memory and has a latency requirement of 100 ms. Which instance type and deployment configuration should the engineer choose to optimize cost while meeting requirements?

Accepted Answer

ml.m5.large (2 vCPU, 8 GB memory). The ml.m5.large instance provides 8 GB of memory, which comfortably exceeds the 4 GB requirement, and its general-purpose compute is sufficient for a model with a 100 ms latency target that does not need GPU acceleration. It is the most cost-effective option among those that meet the memory requirement, since ml.c5.large has only 4 GB (borderline and risky with overhead) and ml.p3.2xlarge is far more expensive with an unnecessary GPU. SageMaker Serverless Inference is not ideal here because cold starts can exceed the 100 ms latency requirement.

Answer

SageMaker Serverless Inference

Answer

ml.c5.large (2 vCPU, 4 GB memory)

Answer

ml.p3.2xlarge (8 vCPU, 61 GB memory, 1 GPU)

Question 8

A company uses Amazon SageMaker to train a model for fraud detection. The dataset has 1 million samples with 200 features. The data is highly imbalanced (0.1% fraud). The team wants to use a random forest model. Which technique should they use to handle the class imbalance during training?

Accepted Answer

Synthetic Minority Over-sampling Technique (SMOTE). SMOTE (Synthetic Minority Over-sampling Technique) generates synthetic samples of the minority class by interpolating between existing minority instances, which helps balance the dataset and improves the random forest's ability to learn the minority class. With only 0.1% fraud out of 1 million samples (1,000 fraud cases), SMOTE increases the effective minority representation without simply duplicating data, reducing overfitting compared to random oversampling.

Answer

Use class weights inversely proportional to class frequencies

Answer

Random undersampling of the majority class

Answer

Adjust the decision threshold after training

Question 9

A machine learning engineer is evaluating a classification model that predicts whether a transaction is fraudulent. The model outputs a probability score. The cost of a false negative (missed fraud) is 10 times higher than the cost of a false positive (false alarm). Which TWO evaluation metrics should the engineer use to tune the model? (Choose TWO.)

Accepted Answer

F-beta score with beta = 2. Option A (F-beta score with beta = 2) is correct because beta > 1 weights recall more heavily than precision, which directly matches the scenario where false negatives (missed fraud) cost 10 times more than false positives; F2 is the standard way to encode that asymmetric cost into a single tunable metric. Option E (Precision-Recall curve) is correct because with a rare positive class like fraud, the PR curve and its area (average precision) focus on the minority class and let the engineer pick a threshold that maximizes recall while keeping precision acceptable, which is exactly the trade-off dictated by the 10:1 cost ratio. Option B (Accuracy) is wrong because with highly imbalanced fraud data a model can score high accuracy by predicting 'not fraud' almost always, hiding the costly false negatives. Option C (Log loss) is wrong because it measures the calibration of predicted probabilities rather than the classification decision threshold tied to the asymmetric misclassification cost. Option D (ROC-AUC) is wrong because it aggregates performance over all thresholds using TPR and FPR, which is insensitive to the class imbalance and does not directly reflect the 10x penalty on false negatives.

Answer

Accuracy

Answer

Log loss

Answer

ROC-AUC

Question 10

A company is deploying a machine learning model using Amazon SageMaker. The model requires GPUs for inference. Which THREE configurations can the company use to meet this requirement? (Choose THREE.)

Accepted Answer

Real-time endpoints with ml.p3 instance types. Option B is correct because real-time endpoints support GPU-backed instance types such as ml.p3 (NVIDIA V100), which are designed for low-latency GPU inference. Option C is correct because Batch Transform can be configured to run on GPU instances like ml.p3, allowing large-scale offline inference jobs that require GPU acceleration. Option E is correct because SageMaker Elastic Inference attaches GPU-powered acceleration to supported instances, providing fractional GPU inference capacity for models that need GPU compute. Option A is not suitable because Serverless Inference does not support GPU instances and runs on CPU-only infrastructure. Option D is incorrect because SageMaker Studio is an integrated development environment (IDE) for building and managing ML workflows, not an inference hosting configuration.

Answer

SageMaker Serverless Inference

Answer

SageMaker Studio

Question 11

A data scientist wants to build a binary classifier to predict customer churn. The dataset has 10,000 records with 500 churners (5%). Which technique should the data scientist use to address class imbalance?

Accepted Answer

Use SMOTE (Synthetic Minority Over-sampling Technique) to create synthetic samples.. SMOTE is the correct choice because it generates synthetic minority-class samples by interpolating between existing minority instances and their nearest neighbors, increasing the minority representation without discarding valuable majority-class data. With only 500 churners out of 10,000 records (5%), SMOTE balances the class distribution while preserving all information from the majority class, which is critical for a churn model where non-churner patterns also carry signal. This typically improves recall and F1 for the minority class compared to simple resampling.

Answer

Randomly undersample the majority class.

Answer

Assign higher class weights to the minority class.

Answer

Downsample the majority class to match the minority class size.

Question 12

A data scientist is tuning a gradient boosting model using SageMaker automatic model tuning. The hyperparameter 'num_round' ranges from 50 to 500. The tuning job uses 'ObjectiveMetric' = 'validation:auc'. After 50 training jobs, the best objective value is 0.95. The data scientist suspects overfitting. What should the data scientist do?

Accepted Answer

Add an early stopping round and increase the range for regularization hyperparameters like 'gamma' and 'lambda'.. The best objective value of 0.95 after 50 jobs with a wide num_round range strongly suggests the model is memorizing training data rather than generalizing. Adding an early stopping round halts training when validation AUC stops improving, and expanding the ranges for regularization hyperparameters (gamma, lambda) lets the tuner find stronger penalties that reduce overfitting. Together these directly address the overfitting symptom while keeping the tuning job productive.

Answer

Increase 'max_depth' to capture more complex patterns.

Answer

Increase 'num_round' to 1000 and keep other hyperparameters unchanged.

Answer

Decrease the range of 'num_round' to 10-100.

Question 13

A company uses Amazon SageMaker to train a linear regression model. The training data includes a feature 'age' with values ranging from 0 to 100. The model's loss is not converging. What is the MOST likely cause?

Accepted Answer

The features are not normalized.. Normalizing features (e.g., scaling 'age' to a similar range as other features) is critical for linear regression convergence. With 'age' ranging 0-100, if other features are in smaller ranges, the optimizer may struggle. Option B (outliers) can cause issues but is less likely than lack of normalization. Option C (instance type too small) typically affects training time, not convergence. Option D (learning rate too high) can cause divergence, but normalization is a more common first step.

Answer

There are outliers in the target variable.

Answer

The instance type is too small.

Answer

The learning rate is too high.

Question 14

A data scientist is building a binary classifier using logistic regression. The dataset has 10 features and 100,000 observations. The model achieves 99% accuracy on the test set, but the precision is 50% and recall is 90%. Which TWO actions should the data scientist take to improve model performance? (Choose 2.)

Accepted Answer

Adjust the decision threshold to increase precision at the cost of recall.. The model has high recall (90%) but low precision (50%), indicating many false positives. Two effective approaches are: adjusting the decision threshold (Option B) to require a higher predicted probability for the positive class, which reduces false positives and increases precision at the cost of some recall. Switching to a random forest classifier (Option C) can capture complex interactions and non-linearities, often improving precision by better separating classes. Option A: Increasing regularization strength (i.e., decreasing C in logistic regression) may help reduce overfitting but does not directly target precision; it may marginally help but is not a primary action. Option D: Collecting more data does not address the underlying class separation issue; it might even amplify imbalance. Option E: Removing features with low correlation could discard valuable information and worsen performance.

Answer

Increase the regularization strength (C) in logistic regression.

Answer

Collect more training data.

Answer

Remove features that have low correlation with the target.

Question 15

A company is deploying a machine learning model for real-time fraud detection. The model must have extremely low latency (<10 ms) and high throughput. Which THREE design choices meet these requirements? (Choose 3.)

Accepted Answer

Use GPU instances (e.g., ml.p3) for the endpoint.. Option A is correct because GPU instances such as ml.p3 provide hardware acceleration that reduces inference latency and increases throughput for compute-intensive ML models, which is essential for sub-10 ms real-time fraud detection. Option D is correct because SageMaker multi-model endpoints host multiple models on the same instance and load them dynamically into memory, improving resource utilization and throughput while keeping latency low for real-time inference. Option E is correct because SageMaker Elastic Inference attaches GPU acceleration to a CPU instance, providing a cost-effective way to lower inference latency and boost throughput without provisioning a full GPU instance. Option B is not correct because using one endpoint per model does not inherently reduce latency or increase throughput and can waste resources. Option C is not correct because SageMaker Batch Transform is designed for offline, batch predictions and is not suitable for real-time, low-latency inference.

Answer

Use one endpoint per model to avoid interference.

Answer

Use SageMaker Batch Transform for real-time predictions.