MLS-C01 Modeling Practice Test 2 — 15 Questions

Question 1

Match each AWS AI service to its capability.

Accepted Answer

Amazon Comprehend: Extract insights and relationships from text. The correct matches are: Amazon Comprehend for extracting insights from text, Amazon Lex for building conversational interfaces, and Amazon Polly for text-to-speech. Distractors swap capabilities between services.

Answer

Amazon Lex: Translate text between languages

Answer

Amazon Polly: Convert speech to text

Question 2

A data scientist is training a binary classification model on a highly imbalanced dataset where the positive class represents only 1% of the data. Which metric should be used to evaluate model performance during training to ensure the model is learning to detect the positive class?

Accepted Answer

F1 score. The F1 score is the harmonic mean of precision and recall, providing a balanced measure that considers both false positives and false negatives. On a highly imbalanced dataset, accuracy is misleading, and F1 score ensures the model is effectively detecting the positive class without being overwhelmed by the majority class.

Answer

Accuracy

Answer

Precision

Answer

Recall

Question 3

A company uses Amazon SageMaker to train a classification model. The training job fails with an error indicating that the algorithm requires a GPU but the instance type does not have one. The scientist used the built-in XGBoost algorithm. What should the scientist do to resolve the issue?

Accepted Answer

Choose a CPU instance type for the training job. SageMaker's built-in XGBoost algorithm is a CPU-based implementation and does not require a GPU — the error is triggered because the training job was launched on an instance type that the algorithm's container cannot use, or the job was misconfigured. The correct fix is to select a CPU instance type (e.g., ml.m5 or ml.c5 family) that matches the built-in XGBoost container's requirements. Built-in XGBoost in SageMaker is designed to run on CPU instances and does not support GPU acceleration out of the box.

Answer

Install a GPU-enabled version of XGBoost in the training container

Answer

Change the algorithm to a deep learning algorithm

Answer

Use a larger GPU instance type

Question 4

A company uses Amazon SageMaker to train a model using the built-in Linear Learner algorithm. The training data contains missing values in some features. What is the best practice for handling missing values with this algorithm?

Accepted Answer

Impute missing values using mean or median imputation. Mean or median imputation is the best practice because Linear Learner expects numeric input and cannot handle missing values natively; imputing with mean or median preserves the dataset size and avoids bias introduced by dropping rows. Median is preferred for skewed distributions, while mean works for roughly symmetric data. This approach is standard in SageMaker's own documentation and examples for Linear Learner.

Answer

Remove rows with missing values

Answer

Set missing values to zero

Answer

Use the `handle_missing` parameter in the algorithm

Question 5

Which TWO metrics are appropriate for evaluating a binary classification model when the cost of false negatives is high?

Accepted Answer

Recall. When false negatives are costly, we want to minimize them, so recall (true positive rate) is important. Precision is also important to avoid too many false positives, but F1 score balances both. Recall directly measures false negatives, and F1 combines precision and recall. AUC-ROC is a general measure, and accuracy can be misleading. Therefore, the two appropriate metrics are Recall (option C) and F1 score (option D).

Answer

Accuracy

Answer

AUC-ROC

Answer

Precision

Question 6

Which TWO actions are best practices for tuning hyperparameters using Amazon SageMaker Automatic Model Tuning?

Accepted Answer

Use Bayesian optimization strategy. Option C is correct because SageMaker Automatic Model Tuning's Bayesian optimization strategy is the default and most efficient approach: it builds a probabilistic model of the objective metric as a function of hyperparameters and uses an acquisition function to pick the next hyperparameter set, converging to good values with fewer training jobs than uninformed search. Option E is correct because SageMaker supports the random search strategy, which samples hyperparameter combinations randomly from the specified ranges; it is a recommended best practice when the search space is large or high-dimensional, when the objective is noisy, or when you want a simple, parallelizable baseline that avoids the sequential nature of Bayesian optimization. Option A is not a best practice because setting the number of training jobs to a very large value wastes compute budget and cost without guaranteeing better results; you should set a reasonable max_jobs based on the search space and budget. Option B is not a best practice because reusing the baseline model's hyperparameters defeats the purpose of tuning — you must define ranges (via IntegerParameter, ContinuousParameter, or CategoricalParameter) for the algorithm's tunable hyperparameters. Option D is not a best practice because SageMaker Automatic Model Tuning does not offer a grid search strategy; grid search is exhaustive and scales poorly with the number of hyperparameters and values.

Answer

Set the number of training jobs to a very large value

Answer

Use the same hyperparameters as the baseline model

Answer

Use grid search strategy

Question 7

Refer to the exhibit. A data scientist runs the above AWS CLI command to create a SageMaker training job using the built-in Linear Learner algorithm. The training job fails with an error. What is the most likely cause?

Accepted Answer

The S3 data type is AugmentedManifestFile, but Linear Learner requires RecordIO or CSV. The most likely cause is that the S3 data type is AugmentedManifestFile, but the Linear Learner algorithm requires RecordIO or CSV format. SageMaker built-in algorithms have specific input format requirements, and Linear Learner supports RecordIO-Protobuf or CSV, not AugmentedManifestFile. This mismatch causes the training job to fail.

Answer

The IAM role does not have sufficient permissions

Answer

The instance type ml.m5.large does not support the Linear Learner algorithm

Answer

The MaxRuntimeInSeconds is too short

Question 8

Refer to the exhibit. A data scientist is configuring SageMaker Model Monitor for data quality checks. The configuration above is used. What is the purpose of the `ProbabilityThresholdAttribute` set to "0.5"?

Accepted Answer

It defines the probability threshold used to convert model output to binary predictions for monitoring. In SageMaker Model Monitor, the ProbabilityThresholdAttribute is used when the model outputs a probability score and you want to convert it into a binary classification (0 or 1) for monitoring purposes. Setting it to 0.5 means any predicted probability >= 0.5 is treated as class 1, and below 0.5 as class 0. This allows Model Monitor to compute classification metrics like accuracy, precision, and recall against ground truth labels.

Answer

It filters the input data to only include predictions above the threshold

Answer

It specifies the threshold for sampling data for monitoring

Answer

It sets the threshold for the accuracy metric

Question 9

A company is building a recommendation system using collaborative filtering on Amazon SageMaker. The dataset contains user-item interactions with a long-tail distribution: a few items have millions of interactions, while most items have very few. The model currently uses matrix factorization with ALS. The recall@20 metric is low for niche items. Which modification would most likely improve recall for long-tail items?

Accepted Answer

Use implicit feedback with confidence weighting to downweight popular items. Using implicit feedback with confidence weighting to downweight popular items is most likely to improve recall for long-tail items. In collaborative filtering, the long-tail distribution means popular items dominate the interactions, causing the model to focus on them and perform poorly on niche items. Implicit feedback with confidence weighting (e.g., the ALS algorithm in Spark) allows the model to assign higher confidence to observed interactions and lower confidence to unobserved ones, and by downweighting popular items, it can better capture niche item preferences. This approach directly addresses the imbalance in the data.

Answer

Increase the regularization parameter to prevent overfitting

Answer

Add explicit features like item category and user demographics

Answer

Increase the number of latent factors in the matrix

Question 10

A company is using Amazon SageMaker to deploy a model that predicts customer churn. The model was trained using a linear learner algorithm. During inference, the endpoint returns predictions that are always 0.5 (the probability of churn). What is the most likely cause?

Accepted Answer

The learning rate is set too high, causing the model to converge to the mean prediction. When the learning rate is too high, the linear learner's optimization can fail to converge to a meaningful decision boundary and instead collapse to predicting the base rate (the mean of the target). For a binary churn problem with roughly balanced classes, that mean is ~0.5, so every inference returns 0.5. This is a classic symptom of an unstable/diverged optimization, not of class imbalance or missing normalization.

Answer

The dataset is highly imbalanced, and the model is predicting the majority class

Answer

The model was trained with too few epochs

Answer

The input features are not normalized

Question 11

A data scientist is using Amazon SageMaker to train a deep learning model for image classification. The training job is using a single GPU instance and is taking too long. The scientist wants to reduce training time without sacrificing model accuracy. The dataset contains 100,000 images of size 256x256. Which change would most effectively reduce training time?

Accepted Answer

Use a smaller image size (e.g., 128x128). Reducing image size from 256x256 to 128x128 cuts the pixel count by 75%, which directly reduces the compute per image and memory bandwidth during forward and backward passes, yielding a substantial training speedup. For many image classification tasks, 128x128 retains enough signal for good accuracy, making it the most effective single change here.

Answer

Reduce the batch size

Answer

Increase the learning rate

Answer

Switch to a distributed training setup with multiple GPUs

Question 12

A company uses Amazon SageMaker to build a text classification model using a pre-trained BERT model. The dataset contains 10,000 labeled documents. The model is overfitting: training accuracy is 99%, validation accuracy is 85%. Which TWO of the following are most likely to help reduce overfitting? (Choose TWO.)

Accepted Answer

Increase the dropout rate during fine-tuning. Option B is correct because increasing the dropout rate during fine-tuning randomly deactivates more neurons, which regularizes the network and reduces its ability to memorize the 10,000 training documents, thereby narrowing the 99% vs 85% train-validation gap. Option E is correct because decreasing the learning rate makes fine-tuning updates smaller and more stable, preventing the model from aggressively fitting noise in the training set and helping it generalize better to validation data. Option A is not appropriate because adding more transformer layers increases model capacity, which would worsen overfitting on a relatively small labeled dataset. Option C is not appropriate because increasing batch size mainly affects training throughput and gradient noise; it does not directly regularize the model and can even reduce the regularizing effect of stochastic gradients. Option D is not appropriate because using a larger pre-trained BERT model adds parameters and capacity, which typically increases overfitting risk rather than reducing it.

Answer

Add more transformer layers to the model

Answer

Increase the batch size

Answer

Use a larger pre-trained BERT model

Question 13

A company is using Amazon SageMaker to train a deep learning model. The training job is taking a long time, and the data scientist wants to reduce training time without sacrificing accuracy. Which technique should they use?

Accepted Answer

Use a larger instance type. Using a larger instance type (more GPUs or CPUs and memory) increases the compute throughput available to the training job, allowing each epoch to complete faster without changing the model architecture or the number of epochs. This reduces wall-clock training time while preserving accuracy because the optimization algorithm and data are unchanged. It is the most direct way to speed up training without sacrificing accuracy.

Answer

Use a smaller instance type

Answer

Reduce the number of epochs

Answer

Use managed spot training with checkpointing

Question 14

A company is building a multiclass classification model using Amazon SageMaker. The dataset has 100 classes and is highly imbalanced. The model currently achieves high accuracy on the majority classes but poor performance on minority classes. Which technique should the data scientist use to improve minority class performance?

Accepted Answer

Use class weights to penalize misclassifications of minority classes. Using class weights to penalize misclassifications of minority classes directly addresses the imbalance by assigning a higher cost to errors on minority classes during training. This forces the model to focus more on learning the minority class patterns, improving recall and F1-score for those classes without altering the dataset distribution. It is a standard technique in SageMaker's built-in algorithms and frameworks like XGBoost and scikit-learn, where the 'scale_pos_weight' or 'class_weight' parameter can be set. This approach is particularly effective for multiclass problems with 100 classes, as it scales the loss contribution per class.

Answer

Apply random oversampling with replacement

Answer

Apply principal component analysis (PCA)

Answer

Remove samples from majority classes

Question 15

A machine learning engineer is training a model using Amazon SageMaker. The training data is stored in S3 and is 10 TB. The engineer wants to use Pipe input mode to stream data from S3. Which algorithms support Pipe mode? (Select all that apply)

Accepted Answer

Amazon SageMaker Linear Learner. Amazon SageMaker's built-in algorithms, including Linear Learner, K-Means, XGBoost, and PCA, all support Pipe mode for streaming training data from S3. Therefore, all the listed options (A, B, C, D) are correct.