MLS-C01 Modeling Practice Test 5 — 15 Questions

Question 1

A data scientist is training a k-means clustering model on a dataset with 1,000 points. The scientist uses the elbow method to choose the number of clusters. The elbow plot shows a clear bend at k=4. After running k-means with k=4, the scientist wants to evaluate the quality of the clustering. Which THREE of the following are suitable internal clustering validation metrics? (Choose THREE.)

Accepted Answer

Calinski-Harabasz index. Silhouette score, Davies-Bouldin index, and Calinski-Harabasz index are all internal validation metrics that do not require ground truth labels. They measure compactness and separation. Rand index and adjusted Rand index require ground truth labels (external validation).

Answer

Adjusted Rand index

Answer

Rand index

Question 2

A data scientist is training a binary classification model on imbalanced data (95% negative, 5% positive). The model achieves 99% accuracy on the test set but fails to detect any positive cases. Which metric should the scientist focus on to evaluate model performance?

Accepted Answer

Recall. Option B is correct because recall (true positive rate) measures the ability to find all positive samples, which is critical for imbalanced datasets where accuracy can be misleading. Option A is wrong because accuracy is high but misleading. Option C is wrong because precision alone doesn't capture the missed positives. Option D is wrong because RMSE is for regression.

Answer

Accuracy

Answer

RMSE

Answer

Precision

Question 3

A company is using Amazon SageMaker to train a deep learning model. The training job is taking a long time, and the data scientist wants to reduce training time without sacrificing accuracy. Which technique should they use?

Accepted Answer

Use managed spot training with checkpointing. Option D is correct because managed spot training can reduce cost and training time by using preemptible instances, often with checkpointing. Option A is wrong because increasing instance count may not linearly reduce time and could increase cost. Option B is wrong because reducing epochs may sacrifice accuracy. Option C is wrong because using a smaller instance is likely to increase time.

Answer

Use a larger instance type

Answer

Use a smaller instance type

Answer

Reduce the number of epochs

Question 4

A machine learning engineer is deploying a model for real-time inference using Amazon SageMaker. The model is a large ensemble that requires 8 GB of memory and 4 vCPUs. The expected traffic is 100 requests per second with a 200 ms latency requirement. Which instance configuration should they choose?

Accepted Answer

ml.c5.2xlarge (8 vCPU, 16 GB). Option C is correct because ml.c5.2xlarge has 8 vCPUs and 16 GB memory, which is sufficient and cost-effective for the workload. Option A is wrong because ml.m5.large has only 2 vCPUs and 8 GB memory, which may not handle the throughput. Option B is wrong because ml.p3.2xlarge is GPU-optimized and overkill. Option D is wrong because ml.t2.medium is too small for the memory requirement.

Answer

ml.t2.medium (2 vCPU, 4 GB)

Answer

ml.p3.2xlarge (8 vCPU, 61 GB GPU)

Answer

ml.m5.large (2 vCPU, 8 GB)

Question 5

A data scientist is using Amazon SageMaker to train a linear regression model. The training data has 10 features, and the scientist wants to interpret the model's coefficients. Which algorithm should they use?

Accepted Answer

Amazon SageMaker Linear Learner. Option A is correct because Linear Learner provides interpretable coefficients. Option B is wrong because XGBoost is tree-based and less interpretable. Option C is wrong because K-Means is unsupervised. Option D is wrong because Factorization Machines are for high-dimensional sparse data.

Answer

Amazon SageMaker XGBoost

Answer

Amazon SageMaker K-Means

Answer

Amazon SageMaker Factorization Machines

Question 6

A company is building a multiclass classification model using Amazon SageMaker. The dataset has 100 classes and is highly imbalanced. The model currently achieves high accuracy on the majority classes but poor performance on minority classes. Which technique should the data scientist use to improve minority class performance?

Accepted Answer

Use class weights to penalize misclassifications of minority classes. Option C is correct because class weights penalize errors on minority classes more, improving their recall. Option A is wrong because removing majority classes would lose data. Option B is wrong because oversampling without replacement may cause overfitting. Option D is wrong because principal component analysis (PCA) is dimensionality reduction, not for imbalance.

Answer

Apply random oversampling with replacement

Answer

Apply principal component analysis (PCA)

Answer

Remove samples from majority classes

Question 7

A data scientist is training an LSTM model for time series forecasting using Amazon SageMaker. The model is overfitting. Which action is LEAST likely to reduce overfitting?

Accepted Answer

Increase the number of LSTM layers. Option D is correct because increasing the number of LSTM layers typically increases model complexity, which can worsen overfitting. Option A is wrong because dropout helps. Option B is wrong because reducing hidden units reduces complexity. Option C is wrong because early stopping prevents overfitting.

Answer

Add dropout layers

Answer

Reduce the number of hidden units

Answer

Use early stopping

Question 8

A company is using Amazon SageMaker to deploy a model for real-time inference. The model requires 500 MB of memory and has a latency requirement of 100 ms. The endpoint is receiving 10 requests per second. Which instance type should be chosen for cost-effectiveness?

Accepted Answer

ml.m5.large. Option A is correct because ml.m5.large (2 vCPU, 8 GB) is more than sufficient for memory and throughput, and is cost-effective. Option B is wrong because ml.c5.xlarge (4 vCPU, 8 GB) is more expensive than needed. Option C is wrong because ml.t2.medium (2 vCPU, 4 GB) has enough memory but may have burstable CPU limitations. Option D is wrong because ml.p3.2xlarge is GPU-optimized and overkill.

Answer

ml.c5.xlarge

Answer

ml.t2.medium

Answer

ml.p3.2xlarge

Question 9

A data scientist is using Amazon SageMaker to train a model. The training job fails with an error 'Insufficient instance capacity'. Which action should the scientist take to resolve this?

Accepted Answer

Retry with a different instance type or Availability Zone. Option D is correct because retrying with a different instance type or in a different Availability Zone often resolves capacity issues. Option A is wrong because the error is about capacity, not permissions. Option B is wrong because reducing data size doesn't affect capacity. Option C is wrong because increasing the instance count may exacerbate the problem.

Answer

Check IAM permissions

Answer

Increase the number of instances

Answer

Reduce the training data size

Question 10

A machine learning engineer is training a model using Amazon SageMaker. The training data is stored in S3 and is 10 TB. The engineer wants to use Pipe input mode to stream data from S3. Which algorithm supports Pipe mode?

Accepted Answer

Amazon SageMaker XGBoost. Option A is correct because Amazon SageMaker XGBoost supports Pipe mode for streaming. Option B is wrong because Linear Learner does not support Pipe mode. Option C is wrong because K-Means does not support Pipe mode. Option D is wrong because PCA does not support Pipe mode.

Answer

Amazon SageMaker Linear Learner

Answer

Amazon SageMaker K-Means

Answer

Amazon SageMaker PCA

Question 11

A data scientist is training a model using Amazon SageMaker. The training job is running on GPU instances, but the GPU utilization is low. Which TWO actions could improve GPU utilization?

Accepted Answer

Use a larger instance with multiple GPUs. Option A is correct because increasing batch size can better utilize GPU parallelism. Option C is correct because using a larger instance with more GPUs can improve utilization. Option B is wrong because decreasing batch size would reduce utilization. Option D is wrong because using CPU instances would not utilize GPU. Option E is wrong because adding more epochs does not affect utilization per step.

Answer

Increase the number of epochs

Answer

Switch to CPU instances

Answer

Decrease the batch size

Question 12

A company is using Amazon SageMaker to deploy a model for real-time inference. The model is a deep neural network that requires GPU for low latency. The endpoint currently uses a single ml.p3.2xlarge instance. Traffic is expected to increase by 5x. Which TWO actions should the company take to handle the increased traffic?

Accepted Answer

Use a larger instance type with more GPUs. Option B is correct because enabling auto-scaling allows the endpoint to handle variable traffic. Option D is correct because using a larger instance with more GPUs (e.g., ml.p3.8xlarge) can increase throughput. Option A is wrong because switching to CPU would increase latency. Option C is wrong because adding more instances without scaling policy may not be optimal. Option E is wrong because reducing batch size would decrease throughput.

Answer

Switch to a CPU-based instance

Answer

Use a multi-model endpoint

Answer

Decrease the batch size

Question 13

A data scientist is performing feature engineering for a machine learning model. The dataset contains categorical features with high cardinality. Which THREE techniques are appropriate for encoding high-cardinality categorical features?

Accepted Answer

Target encoding. Option A is correct because target encoding replaces categories with the mean target value. Option B is correct because count encoding replaces categories with frequency counts. Option D is correct because one-hot encoding can be used if the number of categories is manageable after pruning. Option C is wrong because label encoding implies ordinal relationship, not ideal for nominal high cardinality. Option E is wrong because binary encoding is another option, but the question asks for THREE; typically target, count, and one-hot are common.

Answer

Binary encoding

Answer

Label encoding

Question 14

Refer to the exhibit. A training job failed with the error shown. What is the most likely cause?

Accepted Answer

The training data contains missing values or outliers that cause numerical instability. Option A is correct because the error explicitly states input contains NaN or infinity, indicating missing or invalid values in the training data. Option B is wrong because the error is about input values, not about memory. Option C is wrong because the error is from the training script, not from SageMaker capacity. Option D is wrong because the error is about input data, not about model architecture.

Answer

The model architecture is incorrect

Answer

The instance type does not have enough memory

Answer

The training job exceeded the maximum runtime

Question 15

Refer to the exhibit. A data scientist is trying to create a SageMaker training job but receives an access denied error. The IAM policy shown is attached to the user. What is the likely issue?

Accepted Answer

The policy is missing s3:ListBucket permission. Option C is correct because the policy allows s3:GetObject and s3:PutObject on the bucket, but the training job also needs s3:ListBucket to read the objects. Option A is wrong because the policy has sagemaker:CreateTrainingJob on all resources. Option B is wrong because the policy allows sagemaker actions. Option D is wrong because the policy allows s3:PutObject, but the error is about access denied, not about upload.

Answer

The policy does not allow sagemaker:CreateTrainingJob

Answer

The policy does not allow sagemaker:CreateModel

Answer

The policy does not allow s3:PutObject