MLS-C01 Modeling Practice Test 39 — 15 Questions

Question 1

A data scientist is training a neural network for image classification. The dataset has 50,000 images across 100 classes. The model uses a ResNet-50 architecture pre-trained on ImageNet. The training loss decreases rapidly, but validation loss starts to increase after 5 epochs. Which of the following is the most effective technique to address this?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use data augmentation to increase the diversity of the training set. The rapid decrease in training loss followed by an increase in validation loss after only 5 epochs is a classic sign of overfitting. Data augmentation artificially expands the training set by applying random transformations (e.g., rotations, flips, crops) to existing images, which improves the model's generalization and reduces overfitting. This is the most effective technique among the options because it directly addresses the lack of diverse training examples without changing the model architecture or training hyperparameters in a way that could destabilize learning.

Answer

Increase the learning rate

Answer

Add more layers to the network

Answer

Use a smaller batch size

Question 2

A data scientist is using Amazon SageMaker to deploy a model for real-time inference. The model is a TensorFlow neural network. The scientist wants to use automatic scaling based on the number of incoming requests. Which service integration is required?

Accepted Answer

Amazon SageMaker endpoint configured with Application Auto Scaling. Amazon SageMaker endpoints natively integrate with Application Auto Scaling to adjust the number of instances based on a target metric, such as the number of incoming requests per instance. This allows the TensorFlow model to scale automatically in response to traffic, without needing additional orchestration services.

Answer

Amazon ECS with service auto scaling

Answer

AWS Lambda with provisioned concurrency

Answer

AWS Auto Scaling plans

Question 3

A data scientist is building a text classification model using a bag-of-words approach with logistic regression. The dataset has 10,000 documents and 50,000 unique tokens. The model overfits. Which TWO techniques can help reduce overfitting?

Accepted Answer

Reduce the vocabulary size by removing rare and very frequent terms. Removing rare and very frequent terms reduces the feature space and eliminates noise, which helps the logistic regression model generalize better. Rare terms often act as noise that the model can latch onto for spurious correlations, while very frequent terms (like stopwords) provide little discriminative power. This dimensionality reduction directly combats overfitting by simplifying the model.

Answer

Increase the number of n-grams features

Answer

Use one-hot encoding instead of bag-of-words

Answer

Use a more complex model such as a neural network

Question 4

A company is training a deep learning model for object detection using Amazon SageMaker. The training job is taking too long. Which TWO actions can reduce training time?

Accepted Answer

Use distributed training with multiple GPUs. Distributed training (A) reduces wall-clock time by splitting the workload across multiple GPUs. Using a smaller batch size initially and increasing gradually (warm-up) (D) can help stabilize training and speed convergence by allowing the model to adjust more smoothly early on. Managed spot training (C) reduces cost, not training time, and may increase time due to interruptions. A larger instance (B) does not necessarily reduce training time if the bottleneck is GPU-related, and increasing the number of epochs (E) increases training time.

Answer

Use a larger instance type with more vCPUs

Answer

Use SageMaker managed spot training

Answer

Increase the number of epochs

Question 5

A data scientist is using Amazon SageMaker to train a linear regression model. The training data contains missing values. Which TWO techniques are appropriate for handling missing values in the dataset?

Accepted Answer

Remove rows with missing values if the proportion is small. Removing rows with missing values is a straightforward and effective technique when the proportion of missing data is small (e.g., less than 5% of the total dataset). This avoids introducing bias or distorting the distribution, which is critical for linear regression models that assume complete and normally distributed data. In Amazon SageMaker, this can be done during data preprocessing using built-in transformations or custom scripts before training.

Answer

Use a decision tree model that can handle missing values internally

Answer

Set missing values to zero

Answer

Create a separate category for missing values

Question 6

A company runs an e-commerce platform on AWS. They have a SageMaker endpoint serving a product recommendation model. The model uses a custom container with a TensorFlow model. Recently, the endpoint has been returning high latency and occasional 504 errors during peak traffic. The data scientist observes that the model inference time is around 200 ms per request, but the endpoint is configured with a single ml.c5.large instance. The traffic spikes can reach 100 requests per second. The data scientist needs to reduce latency and eliminate 504 errors. Which course of action is most appropriate?

Accepted Answer

Configure the SageMaker endpoint with Application Auto Scaling to scale out based on the 'InvocationsPerInstance' metric, and use a larger instance type such as ml.c5.xlarge. The endpoint is bottlenecked by both instance size and concurrency. With a single ml.c5.large instance handling 100 requests per second and a 200 ms inference time, the instance can only process about 5 requests per second (1000 ms / 200 ms = 5 requests per second per instance). Application Auto Scaling based on the 'InvocationsPerInstance' metric will add instances during traffic spikes, while upgrading to ml.c5.xlarge doubles compute capacity per instance, reducing latency and eliminating 504 errors caused by request queue overflow.

Answer

Use Amazon Elastic Inference to attach an EI accelerator to the endpoint instance

Answer

Switch to a multi-model endpoint to serve multiple models on the same instance

Answer

Replace the SageMaker endpoint with an AWS Lambda function that loads the model from S3 and returns predictions

Question 7

A data scientist is working on a binary classification problem to predict loan default. The dataset has 200,000 samples and 50 features. The target variable is imbalanced: 5% default, 95% non-default. The scientist trains a logistic regression model and achieves 95% accuracy, but the recall for the default class is only 20%. The business requires that at least 70% of actual defaults be identified (recall >= 0.7). Which approach should the scientist take to improve recall without significantly sacrificing precision?

Accepted Answer

Use oversampling techniques like SMOTE to create synthetic samples of the minority class. SMOTE (Synthetic Minority Oversampling Technique) creates synthetic samples for the minority class by interpolating between existing minority instances, which increases the representation of the default class in the training data. This directly addresses the low recall (20%) by providing the logistic regression model with more balanced class distributions, enabling it to learn decision boundaries that capture more true positives without discarding majority class information. Unlike simple oversampling, SMOTE reduces overfitting risk by generating novel samples rather than duplicating existing ones, which helps maintain precision while improving recall.

Answer

Use random undersampling of the majority class to balance the dataset

Answer

Change the decision threshold to 0.3

Answer

Increase the regularization strength (C) in logistic regression

Question 8

A company is using Amazon SageMaker to train a XGBoost model for predicting customer churn. The training data is stored in an S3 bucket as CSV files. The data scientist runs a hyperparameter tuning job with 50 training jobs. The tuning job completes, but the best model's accuracy on the holdout set is lower than expected. The data scientist suspects that the hyperparameter ranges are too narrow. Which corrective action is most appropriate?

Accepted Answer

Expand the hyperparameter ranges for key parameters such as 'max_depth', 'learning_rate', and 'subsample'. The data scientist suspects the hyperparameter ranges are too narrow, which directly limits the model's ability to find an optimal configuration. Expanding ranges for key XGBoost parameters like 'max_depth', 'learning_rate', and 'subsample' allows the tuning job to explore a broader space of model complexities and regularization levels, potentially improving accuracy on the holdout set. This is the most direct fix for the stated problem, as it addresses the root cause rather than increasing job count or changing the search strategy.

Answer

Increase the number of training jobs in the tuning job

Answer

Switch to a different algorithm like Random Forest

Answer

Change the tuning strategy from random search to Bayesian optimization

Question 9

A data scientist is training a binary classification model on an imbalanced dataset where the positive class represents only 1% of the data. The model achieves 99% accuracy but fails to identify most positive cases. Which metric should the data scientist use to evaluate model performance?

Accepted Answer

F1 score. The F1 score is the harmonic mean of precision and recall, making it ideal for imbalanced datasets where accuracy is misleading. Since the model achieves 99% accuracy by simply predicting the majority class (negative), it fails to capture positive cases; F1 score penalizes this by balancing false positives and false negatives, providing a more truthful performance measure.

Answer

R-squared

Answer

Accuracy

Answer

RMSE

Question 10

A machine learning team is deploying a model for real-time fraud detection. The model must make predictions with less than 100ms latency. The team uses SageMaker and the model is a large ensemble of decision trees. Which SageMaker hosting option is MOST suitable?

Accepted Answer

SageMaker Serverless Inference. SageMaker Serverless Inference is the most suitable option because it automatically scales to handle variable traffic and does not require managing underlying infrastructure. Although it may incur cold starts, for a constantly invoked fraud detection model the endpoint remains warm, achieving sub-100ms latency. The large ensemble of decision trees can be deployed as a single model on a Serverless endpoint, which is optimized for real-time inference with low latency and automatic scaling.

Answer

SageMaker Multi-model endpoint

Answer

SageMaker Elastic Inference

Answer

SageMaker Batch Transform

Question 11

A data scientist is building a regression model to predict house prices. The dataset contains features like 'number_of_rooms' (integer), 'sqft' (float), 'location' (categorical with 1000 unique values). Which feature engineering approach is BEST for the 'location' feature?

Accepted Answer

Target encoding. Target encoding is the best approach for the 'location' feature because it has 1,000 unique categories, making one-hot encoding infeasible (would create 1,000 dummy columns) and label encoding inappropriate (imposes arbitrary ordinal relationships). Target encoding replaces each category with the mean of the target variable (house price) for that category, capturing the predictive signal of location while keeping the feature as a single numeric column. This balances model performance with dimensionality and avoids overfitting when regularized (e.g., with smoothing or cross-validation).

Answer

Remove the feature

Answer

One-hot encoding

Answer

Label encoding

Question 12

A team trained a multiclass classification model using SageMaker built-in XGBoost. The model's accuracy is high, but for a specific class, recall is very low. The team wants to improve recall for that class without significant accuracy drop. Which approach is MOST effective?

Accepted Answer

Resample the training data to balance the class representation. Resampling the training data to balance class representation directly addresses the root cause of low recall for a specific class in a multiclass XGBoost model. XGBoost's built-in objective functions (e.g., 'multi:softmax') optimize for overall accuracy, which can bias the model toward majority classes; resampling (e.g., oversampling the minority class or undersampling the majority) forces the model to learn decision boundaries that better capture the minority class, improving recall without drastically reducing overall accuracy.

Answer

Add more training data from all classes

Answer

Increase the max_depth hyperparameter of XGBoost

Answer

Switch from XGBoost to a linear learner

Question 13

A company uses SageMaker to train a model that processes sensitive customer data. Due to compliance, the training data must be encrypted at rest and in transit, and the model artifacts must be stored in a secured S3 bucket with encryption. Which combination of actions is REQUIRED?

Accepted Answer

Enable AWS KMS encryption for the SageMaker notebook and training job, and use an S3 bucket with default encryption using AWS KMS. It ensures end-to-end encryption: AWS KMS encryption for the SageMaker notebook and training job encrypts data in transit and at rest within the SageMaker environment, while an S3 bucket with default encryption using AWS KMS encrypts the training data and model artifacts at rest. This combination meets compliance requirements for encryption at rest and in transit, as KMS provides envelope encryption with customer-managed keys, and SageMaker automatically uses TLS for data in transit when KMS is enabled.

Answer

Store data in an S3 bucket with AWS CloudHSM integration

Answer

Use an S3 bucket with SSE-S3 and enable SageMaker Internet-facing mode

Answer

Use an S3 bucket with default encryption (SSE-S3) and enable SSL for all connections

Question 14

A data scientist is using SageMaker to train a linear regression model. The target variable has a long-tail distribution. Which data transformation is LEAST likely to improve model performance?

Accepted Answer

Normalize all feature values to [0,1]. Normalizing feature values to [0,1] is a scaling technique that does not address the long-tail distribution of the target variable. Long-tail distributions typically require transformations that compress the tail (e.g., log or Box-Cox) to make the relationship more linear and reduce the influence of extreme values. Feature normalization helps with gradient descent convergence but does not fix skewness in the target, so it is least likely to improve model performance for this specific issue.

Answer

Add interaction terms between features

Answer

Apply log transformation to the target variable

Answer

Remove outliers from the target variable

Question 15

A team is deploying a SageMaker endpoint for a model that was trained with scikit-learn. The endpoint receives spikes in traffic during business hours. The team wants to minimize cost while ensuring availability during spikes. Which endpoint configuration is MOST appropriate?

Accepted Answer

Use a production variant endpoint with auto-scaling based on CPU utilization. A production variant endpoint with auto-scaling based on CPU utilization allows the SageMaker endpoint to dynamically adjust the number of instances in response to traffic spikes, ensuring availability during business hours while minimizing cost by scaling down during off-peak periods. This approach is ideal for a scikit-learn model, which is CPU-bound, making CPU utilization a relevant and effective scaling metric.

Answer

Use SageMaker Serverless Inference

Answer

Use a multi-model endpoint with a single instance type

Answer

Deploy a single large instance that can handle peak load