MLS-C01 Modeling Practice Test 4 — 15 Questions

Question 1

A data scientist is training a text classification model using Amazon SageMaker's built-in BlazingText algorithm. The dataset contains 1 million documents. Which TWO hyperparameters are most important to tune for improving model accuracy?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Learning rate. Option A (Learning rate) is correct because BlazingText's learning rate controls the step size during SGD updates, and tuning it is critical to avoid underfitting (too small) or divergence/overshooting (too large) when training on 1 million documents. Option E (Number of epochs) is correct because it determines how many full passes over the dataset the algorithm makes; with a large corpus, too few epochs leads to underfitting while too many causes overfitting, so tuning it directly affects accuracy. Option B (Batch size) is not a tunable BlazingText hyperparameter exposed for tuning in this context, and it is not among the most impactful for accuracy. Option C (Loss function) is fixed by the algorithm (e.g., softmax cross-entropy for classification) and cannot be tuned. Option D (Type of optimizer) is also not user-selectable in BlazingText, so it cannot be tuned to improve accuracy.

Answer

Batch size

Answer

Loss function

Answer

Type of optimizer

Question 2

A data scientist is training a binary classification model and wants to evaluate its performance using a metric that is robust to class imbalance. Which metric should be used?

Accepted Answer

F1 score. The F1 score is the harmonic mean of precision and recall, so it balances both false positives and false negatives and is not dominated by the majority class. This makes it robust to class imbalance, unlike accuracy, which can be misleadingly high when one class dominates. F1 is the standard choice when both precision and recall matter and classes are skewed.

Answer

Mean squared error

Answer

Area under the ROC curve (AUC)

Answer

Accuracy

Question 3

A data scientist is using Amazon SageMaker to train a model and wants to automatically stop the training job if the loss does not improve for a certain number of epochs. Which SageMaker feature can be used for this purpose?

Accepted Answer

Custom early stopping callback in the training script. SageMaker does not provide a native, framework-agnostic early stopping feature for arbitrary training scripts — early stopping must be implemented inside the training script itself, typically as a callback that monitors validation loss and terminates training when no improvement is seen for N epochs. This is the standard approach for custom training code in SageMaker.

Answer

SageMaker Experiments

Answer

SageMaker Automatic Model Tuning

Answer

SageMaker Debugger

Question 4

A machine learning team is deploying a real-time inference endpoint for a recommendation model using Amazon SageMaker. The model takes a long time to load (several minutes) due to its size (5 GB). Which deployment strategy minimizes the cold start latency?

Accepted Answer

Use Multi-Model Endpoints to keep the model loaded between invocations. Multi-Model Endpoints (MME) allow multiple models to be loaded on the same endpoint and keep them cached in memory between invocations, reducing cold start latency for subsequent calls. This is ideal for large models like the 5 GB recommendation model. Option A (single instance with large memory) does not address load time. Option C (Serverless Inference) incurs cold starts on each invocation. Option D (larger instance type with more vCPUs) may speed up loading but does not prevent cold starts after idle periods. Thus, MMEs minimize cold start by maintaining model persistence.

Answer

Use a single instance with a large memory size

Answer

Use SageMaker Serverless Inference

Answer

Use a larger instance type with more vCPUs

Question 5

A data scientist is training a neural network for time series forecasting. The training loss decreases initially but then starts to increase after 20 epochs. Which action should the scientist take to address this?

Accepted Answer

Implement early stopping based on validation loss. When training loss decreases then increases after ~20 epochs, the model has begun to overfit — it memorizes training data and generalizes worse. Early stopping monitors validation loss and halts training when it stops improving, restoring the best-performing checkpoint. This directly addresses the observed divergence between training and validation performance.

Answer

Increase the dropout rate

Answer

Increase the learning rate

Answer

Add more layers to the network

Question 6

Which TWO of the following are valid techniques for handling missing values in a dataset for machine learning?

Accepted Answer

Remove rows with missing values. Option B (Remove rows with missing values) is a valid and widely used technique known as listwise deletion or complete-case analysis, where any record containing a missing value is dropped so the model trains only on complete observations. Option E (Replace missing values with the mean of the feature) is a standard imputation method called mean imputation, which fills numeric gaps with the feature's arithmetic average to preserve the overall central tendency and avoid losing data. The other options are not standard accepted techniques: replacing values with the maximum (A) is an arbitrary extreme-value substitution that distorts the distribution, adding random noise (C) injects artificial variance unrelated to the true data, and converting missing values to the string 'missing' (D) is only meaningful as a categorical encoding for tree-based models and is not a general-purpose missing-value handling technique.

Answer

Replace missing values with the maximum value of the feature

Answer

Replace missing values with random noise

Answer

Convert missing values to the string 'missing'

Question 7

A data scientist is trying to create a SageMaker training job but receives an access denied error. The IAM policy attached to the role is shown in the exhibit. What is the most likely cause of the error?

Accepted Answer

The policy does not allow s3:PutObject for the output location. SageMaker training jobs write model artifacts and output to an S3 location specified in the job configuration. If the IAM role lacks s3:PutObject permission for that output bucket/prefix, the job fails with access denied. The most likely cause is a missing PutObject permission on the output location.

Answer

The policy does not allow sagemaker:CreateTrainingJob

Answer

The policy has an explicit deny on s3:PutObject

Answer

The policy does not allow s3:GetObject on the output bucket

Question 8

A data scientist ran an XGBoost training job in SageMaker and it failed with the error shown in the exhibit. Which hyperparameter change is most likely to resolve the numeric overflow?

Accepted Answer

Reduce max_depth to a lower value. The error is a numeric overflow, which in XGBoost often occurs when the model computes gradients or Hessians that exceed floating-point limits. Reducing max_depth lowers the complexity of each tree, which reduces the magnitude of leaf weights and the resulting gradients, thereby preventing overflow. This is the most direct hyperparameter change to address the issue.

Answer

Increase subsample

Answer

Increase eta (learning rate)

Answer

Increase num_round

Question 9

A data science team is training a binary classification model using Amazon SageMaker. The dataset is highly imbalanced (95% negative class, 5% positive class). The team wants to maximize the F1 score. Which built-in SageMaker algorithm is most appropriate?

Accepted Answer

XGBoost. XGBoost supports scale_pos_weight to handle class imbalance, directly optimizing for F1. Linear Learner with balanced class weights can also help but typically optimizes log loss. K-Means is unsupervised. PCA is for dimensionality reduction.

Answer

Linear Learner

Answer

PCA

Answer

K-Means

Question 10

A company is using Amazon SageMaker to deploy a model for real-time inference. The model receives requests with varying payload sizes. The company observes occasional latency spikes. Which feature can help mitigate this?

Accepted Answer

Amazon SageMaker Inference Recommender. Amazon SageMaker Inference Recommender is the correct answer because it is specifically designed to help select the optimal instance type and configuration for a model based on performance requirements, including latency and throughput. It runs load tests across different instance types and configurations, providing recommendations that can minimize latency spikes for varying payload sizes. By using Inference Recommender, the company can identify the best deployment configuration to handle varying payload sizes without latency spikes.

Answer

Multi-model endpoints

Answer

Amazon Elastic Inference

Answer

Automatic scaling

Question 11

A company is using Amazon SageMaker to deploy a model for real-time inference. The model is updated frequently. Which deployment strategy allows for zero-downtime updates and easy rollback?

Accepted Answer

Blue/green deployment using endpoint updates. Blue/green deployment using endpoint updates is correct because SageMaker endpoint updates natively support shifting 100% of traffic to a new endpoint (green) while the old endpoint (blue) remains available, enabling zero-downtime cutover and instant rollback by reverting traffic weights. This is the canonical pattern for frequent model updates requiring safe rollback.

Answer

Canary deployment

Answer

A/B testing with production variants

Answer

Multi-model endpoint

Question 12

A data scientist is using Amazon SageMaker to train a model. The training dataset is stored in S3 as CSV files. The scientist wants to use the SageMaker built-in Linear Learner algorithm. Which input mode should be used for optimal performance?

Accepted Answer

Pipe mode. Pipe mode streams data directly from S3 to the algorithm without writing to disk, reducing I/O overhead. File mode downloads the entire dataset to disk, which is slower. Fast file mode is not a SageMaker feature. Augmented manifest is for additional metadata, not performance.

Answer

Augmented manifest file mode

Answer

File mode

Answer

Fast file mode

Question 13

Which TWO of the following are appropriate use cases for Amazon SageMaker built-in algorithms?

Accepted Answer

Classifying customer churn using tabular data. Option A is correct because SageMaker provides built-in tabular classification algorithms such as XGBoost and Linear Learner, which are designed for supervised binary/multiclass problems like predicting customer churn from tabular data. Option C is correct because SageMaker includes built-in natural language processing algorithms like BlazingText (word2vec/fastText) and Object2Vec that generate word embeddings for text classification tasks. Option B is not appropriate because SageMaker built-in algorithms do not include a Q-learning reinforcement learning algorithm; RL in SageMaker typically requires custom code or the RL container with custom toolkits. Option D is not appropriate because custom CNN architectures require bringing your own model or framework (e.g., TensorFlow, PyTorch) rather than a built-in algorithm, though built-in image classification exists for standard transfer learning. Option E is not appropriate because ARIMA is not offered as a SageMaker built-in algorithm; SageMaker provides DeepAR for time series forecasting, while ARIMA would need to be implemented via custom code or another service.

Answer

Reinforcement learning using Q-learning

Answer

Image classification using a custom CNN architecture

Answer

Time series forecasting using ARIMA

Question 14

A data scientist is training a binary classification model on an imbalanced dataset where the positive class represents 1% of the data. The model needs to maximize recall while keeping precision above 0.7. Which sampling strategy should the data scientist use?

Accepted Answer

SMOTE from imbalanced-learn to generate synthetic samples for the minority class.. (SMOTE) is correct because it generates synthetic samples for the minority class, which can improve recall without discarding data, and synthetic samples help maintain precision above 0.7 by providing more balanced training. Option A (NearMiss) undersamples majority samples based on distance, potentially discarding important data and reducing recall. Option C (RandomUnderSampler) may lose too many majority samples, harming recall and precision. Option D (TomekLinks) only removes overlapping samples, which does not sufficiently address imbalance. Option E (RandomOverSampler) duplicates minority samples, which can cause overfitting and reduce precision on unseen data.

Answer

NearMiss from imbalanced-learn to undersample the majority class based on distance to minority samples.

Answer

RandomUnderSampler from imbalanced-learn to undersample the majority class.

Answer

TomekLinks from imbalanced-learn to remove overlapping samples.

Answer

RandomOverSampler from imbalanced-learn to oversample the minority class.

Question 15

A data scientist is training a gradient boosting model on a large dataset (100 GB) stored in Amazon S3. The training job uses a SageMaker built-in XGBoost algorithm with a single ml.p3.2xlarge instance. The job fails with a memory error. Which solution should the data scientist adopt to resolve the memory issue?

Accepted Answer

Increase the number of training instances to use distributed training with XGBoost.. SageMaker's built-in XGBoost algorithm supports distributed training via the 'distributed' or 'fully distributed' modes, which shard the training data across multiple instances and aggregate gradients, allowing the model to fit in memory across the cluster. Increasing the number of instances (horizontal scaling) is the recommended fix when a single ml.p3.2xlarge runs out of memory on a 100 GB dataset. This preserves accuracy because the algorithm still sees the full dataset, just partitioned across workers.

Answer

Reduce the number of features by using PCA before training.

Answer

Use SageMaker Pipe mode to stream data directly from S3 instead of downloading it.

Answer

Use SageMaker BlazingText algorithm with negative sampling.

Answer

Switch to SageMaker Linear Learner algorithm, which requires less memory.