MLS-C01 Modeling Practice Test 39 — 15 Questions

Question 1

A data scientist is building a text classification model using a bag-of-words approach with logistic regression. The dataset has 10,000 documents and 50,000 unique tokens. The model overfits. Which TWO techniques can help reduce overfitting?

Accepted Answer

Reduce the vocabulary size by removing rare and very frequent terms. Option D is correct because removing rare and very frequent terms reduces the feature space and eliminates noise, which helps the logistic regression model generalize better. Rare terms often act as noise that the model can latch onto for spurious correlations, while very frequent terms (like stopwords) provide little discriminative power. This dimensionality reduction directly combats overfitting by simplifying the model.

Answer

Increase the number of n-grams features

Answer

Use one-hot encoding instead of bag-of-words

Answer

Use a more complex model such as a neural network

Question 2

A company is training a deep learning model for object detection using Amazon SageMaker. The training job is taking too long. Which THREE actions can reduce training time?

Accepted Answer

Use distributed training with multiple GPUs. Option A is correct because distributed training with multiple GPUs (e.g., using SageMaker's distributed data parallelism or model parallelism) splits the workload across multiple devices, reducing wall-clock time per epoch. This leverages Horovod or SageMaker's own distributed training libraries to synchronize gradients efficiently, directly addressing the long training time.

Answer

Use a larger instance type with more vCPUs

Answer

Increase the number of epochs

Question 3

A data scientist is using Amazon SageMaker to train a linear regression model. The training data contains missing values. Which TWO techniques are appropriate for handling missing values in the dataset?

Accepted Answer

Remove rows with missing values if the proportion is small. Options A (Impute with the mean of the column) and C (Remove rows with missing values if the proportion is small) are correct. Imputation is a common technique; removing rows is acceptable if few missing. Option B (Set missing values to zero) can bias the model. Option D (Use a decision tree that handles missing values internally) is not applicable for linear regression. Option E (Use a separate category for missing) is not suitable for linear regression numeric features.

Answer

Use a decision tree model that can handle missing values internally

Answer

Set missing values to zero

Answer

Create a separate category for missing values

Question 4

A company runs an e-commerce platform on AWS. They have a SageMaker endpoint serving a product recommendation model. The model uses a custom container with a TensorFlow model. Recently, the endpoint has been returning high latency and occasional 504 errors during peak traffic. The data scientist observes that the model inference time is around 200 ms per request, but the endpoint is configured with a single ml.c5.large instance. The traffic spikes can reach 100 requests per second. The data scientist needs to reduce latency and eliminate 504 errors. Which course of action is most appropriate?

Accepted Answer

Configure the SageMaker endpoint with Application Auto Scaling to scale out based on the 'InvocationsPerInstance' metric, and use a larger instance type such as ml.c5.xlarge. Option B is correct because the endpoint is bottlenecked by both instance size and concurrency. With a single ml.c5.large instance handling 100 requests per second and a 200 ms inference time, the instance can only process about 5 requests per second (1000 ms / 200 ms = 5 requests per second per instance). Application Auto Scaling based on the 'InvocationsPerInstance' metric will add instances during traffic spikes, while upgrading to ml.c5.xlarge doubles compute capacity per instance, reducing latency and eliminating 504 errors caused by request queue overflow.

Answer

Use Amazon Elastic Inference to attach an EI accelerator to the endpoint instance

Answer

Switch to a multi-model endpoint to serve multiple models on the same instance

Answer

Replace the SageMaker endpoint with an AWS Lambda function that loads the model from S3 and returns predictions

Question 5

A data scientist is working on a binary classification problem to predict loan default. The dataset has 200,000 samples and 50 features. The target variable is imbalanced: 5% default, 95% non-default. The scientist trains a logistic regression model and achieves 95% accuracy, but the recall for the default class is only 20%. The business requires that at least 70% of actual defaults be identified (recall >= 0.7). Which approach should the scientist take to improve recall without significantly sacrificing precision?

Accepted Answer

Use oversampling techniques like SMOTE to create synthetic samples of the minority class. SMOTE (Synthetic Minority Oversampling Technique) creates synthetic samples for the minority class by interpolating between existing minority instances, which increases the representation of the default class in the training data. This directly addresses the low recall (20%) by providing the logistic regression model with more balanced class distributions, enabling it to learn decision boundaries that capture more true positives without discarding majority class information. Unlike simple oversampling, SMOTE reduces overfitting risk by generating novel samples rather than duplicating existing ones, which helps maintain precision while improving recall.

Answer

Use random undersampling of the majority class to balance the dataset

Answer

Change the decision threshold to 0.3

Answer

Increase the regularization strength (C) in logistic regression

Question 6

A company is using Amazon SageMaker to train a XGBoost model for predicting customer churn. The training data is stored in an S3 bucket as CSV files. The data scientist runs a hyperparameter tuning job with 50 training jobs. The tuning job completes, but the best model's accuracy on the holdout set is lower than expected. The data scientist suspects that the hyperparameter ranges are too narrow. Which corrective action is most appropriate?

Accepted Answer

Expand the hyperparameter ranges for key parameters such as 'max_depth', 'learning_rate', and 'subsample'. Option C is correct because the data scientist suspects the hyperparameter ranges are too narrow, which directly limits the model's ability to find an optimal configuration. Expanding ranges for key XGBoost parameters like 'max_depth', 'learning_rate', and 'subsample' allows the tuning job to explore a broader space of model complexities and regularization levels, potentially improving accuracy on the holdout set. This is the most direct fix for the stated problem, as it addresses the root cause rather than increasing job count or changing the search strategy.

Answer

Increase the number of training jobs in the tuning job

Answer

Switch to a different algorithm like Random Forest

Answer

Change the tuning strategy from random search to Bayesian optimization

Question 7

A data scientist is training a binary classification model on an imbalanced dataset where the positive class represents only 1% of the data. The model achieves 99% accuracy but fails to identify most positive cases. Which metric should the data scientist use to evaluate model performance?

Accepted Answer

F1 score. The F1 score is the harmonic mean of precision and recall, making it ideal for imbalanced datasets where accuracy is misleading. Since the model achieves 99% accuracy by simply predicting the majority class (negative), it fails to capture positive cases; F1 score penalizes this by balancing false positives and false negatives, providing a more truthful performance measure.

Answer

R-squared

Answer

Accuracy

Answer

RMSE

Question 8

A machine learning team is deploying a model for real-time fraud detection. The model must make predictions with less than 100ms latency. The team uses SageMaker and the model is a large ensemble of decision trees. Which SageMaker hosting option is MOST suitable?

Accepted Answer

SageMaker Multi-model endpoint. A SageMaker Multi-model endpoint is the most suitable option because it allows hosting multiple models (including large ensembles) on a single endpoint while sharing underlying compute resources, which reduces cost and latency. The endpoint dynamically loads and caches models based on inference requests, enabling sub-100ms predictions for large ensemble models by keeping frequently used models in memory.

Answer

SageMaker Serverless Inference

Answer

SageMaker Elastic Inference

Answer

SageMaker Batch Transform

Question 9

A team is training a deep learning model on SageMaker using a custom PyTorch container. Training takes 24 hours on a single ml.p3.2xlarge instance. The team wants to reduce training time using distributed training. Which strategy is MOST appropriate?

Accepted Answer

Use model parallelism to split the model across multiple GPUs. Option C (Model parallelism) is correct for deep learning models that are too large to fit on a single GPU. Option A (Data parallelism) would require the model to fit on each GPU, which may not be the case. Option B (Hyperparameter tuning) does not reduce training time directly. Option D (Spot instances) may cause interruptions and does not guarantee speedup.

Answer

Use data parallelism with Horovod across multiple instances

Answer

Use SageMaker Managed Spot Training to reduce cost

Answer

Use SageMaker Automatic Model Tuning to find optimal hyperparameters

Question 10

A data scientist is building a regression model to predict house prices. The dataset contains features like 'number_of_rooms' (integer), 'sqft' (float), 'location' (categorical with 1000 unique values). Which feature engineering approach is BEST for the 'location' feature?

Accepted Answer

Target encoding. Target encoding is the best approach for the 'location' feature because it has 1,000 unique categories, making one-hot encoding infeasible (would create 1,000 dummy columns) and label encoding inappropriate (imposes arbitrary ordinal relationships). Target encoding replaces each category with the mean of the target variable (house price) for that category, capturing the predictive signal of location while keeping the feature as a single numeric column. This balances model performance with dimensionality and avoids overfitting when regularized (e.g., with smoothing or cross-validation).

Answer

Remove the feature

Answer

One-hot encoding

Answer

Label encoding

Question 11

A team trained a multiclass classification model using SageMaker built-in XGBoost. The model's accuracy is high, but for a specific class, recall is very low. The team wants to improve recall for that class without significant accuracy drop. Which approach is MOST effective?

Accepted Answer

Resample the training data to balance the class representation. Option B is correct because resampling the training data to balance class representation directly addresses the root cause of low recall for a specific class in a multiclass XGBoost model. XGBoost's built-in objective functions (e.g., 'multi:softmax') optimize for overall accuracy, which can bias the model toward majority classes; resampling (e.g., oversampling the minority class or undersampling the majority) forces the model to learn decision boundaries that better capture the minority class, improving recall without drastically reducing overall accuracy.

Answer

Add more training data from all classes

Answer

Increase the max_depth hyperparameter of XGBoost

Answer

Switch from XGBoost to a linear learner

Question 12

A company uses SageMaker to train a model that processes sensitive customer data. Due to compliance, the training data must be encrypted at rest and in transit, and the model artifacts must be stored in a secured S3 bucket with encryption. Which combination of actions is REQUIRED?

Accepted Answer

Enable AWS KMS encryption for the SageMaker notebook and training job, and use an S3 bucket with default encryption using AWS KMS. Option D is correct because it ensures end-to-end encryption: AWS KMS encryption for the SageMaker notebook and training job encrypts data in transit and at rest within the SageMaker environment, while an S3 bucket with default encryption using AWS KMS encrypts the training data and model artifacts at rest. This combination meets compliance requirements for encryption at rest and in transit, as KMS provides envelope encryption with customer-managed keys, and SageMaker automatically uses TLS for data in transit when KMS is enabled.

Answer

Store data in an S3 bucket with AWS CloudHSM integration

Answer

Use an S3 bucket with SSE-S3 and enable SageMaker Internet-facing mode

Answer

Use an S3 bucket with default encryption (SSE-S3) and enable SSL for all connections

Question 13

A data scientist is using SageMaker to train a linear regression model. The target variable has a long-tail distribution. Which data transformation is LEAST likely to improve model performance?

Accepted Answer

Normalize all feature values to [0,1]. Option C (Normalization of features) is least likely to help because linear regression is scale-invariant; normalization does not change the model's performance. Option A (Log transformation) can reduce skewness. Option B (Removing outliers) can improve fit. Option D (Adding interaction terms) can capture relationships.

Answer

Add interaction terms between features

Answer

Apply log transformation to the target variable

Answer

Remove outliers from the target variable

Question 14

A team is deploying a SageMaker endpoint for a model that was trained with scikit-learn. The endpoint receives spikes in traffic during business hours. The team wants to minimize cost while ensuring availability during spikes. Which endpoint configuration is MOST appropriate?

Accepted Answer

Use a production variant endpoint with auto-scaling based on CPU utilization. Option B is correct because a production variant endpoint with auto-scaling based on CPU utilization allows the SageMaker endpoint to dynamically adjust the number of instances in response to traffic spikes, ensuring availability during business hours while minimizing cost by scaling down during off-peak periods. This approach is ideal for a scikit-learn model, which is CPU-bound, making CPU utilization a relevant and effective scaling metric.

Answer

Use SageMaker Serverless Inference

Answer

Use a multi-model endpoint with a single instance type

Answer

Deploy a single large instance that can handle peak load

Question 15

A data scientist is training a model using SageMaker's built-in XGBoost algorithm. The dataset has 500 features and 1 million rows. The training job is taking too long. The scientist wants to reduce training time without sacrificing accuracy. Which action is LIKELY to be most effective?

Accepted Answer

Apply Principal Component Analysis (PCA) to reduce the number of features. Option A (Reduce the number of features by applying PCA) is correct because it reduces dimensionality, speeding up training. Option B (Increase the number of instances) may not be cost-effective. Option C (Use a smaller instance) may reduce time but also accuracy. Option D (Reduce the number of trees) may reduce accuracy.

Answer

Use a smaller instance type to reduce time

Answer

Reduce the number of trees in XGBoost

Answer

Use a larger instance type with more vCPUs