MLS-C01 Practice Exam 54 — 20 Questions

Question 1

A data scientist is working on a regression problem with a dataset that contains outliers. The data scientist is choosing between mean squared error (MSE) and mean absolute error (MAE) as the loss function. Which loss function is more robust to outliers?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

MAE, because it treats all errors equally.. MAE is more robust to outliers because it uses the absolute difference between predicted and actual values, which does not disproportionately penalize large errors. In contrast, MSE squares the errors, causing outliers to have a much larger influence on the loss and model updates. This makes MAE less sensitive to extreme values in regression tasks.

Answer

Both are equally robust to outliers.

Answer

MSE, because it penalizes large errors more heavily.

Answer

Neither is robust; use Huber loss instead.

Question 2

A company uses Amazon SageMaker to train a model. The training job runs successfully but the model artifacts are not saved to the specified S3 output path. What is a likely cause?

Accepted Answer

The training script does not save the model to /opt/ml/model.. Amazon SageMaker expects the training script to save the model artifacts to the `/opt/ml/model` directory. After the training job completes, SageMaker automatically copies the contents of this directory to the specified S3 output path. If the script saves the model elsewhere (e.g., `/tmp` or a custom path), no artifacts will be uploaded, resulting in an empty or missing S3 output.

Answer

The model size exceeds the S3 bucket limit.

Answer

The training job used spot instances.

Answer

The S3 bucket is in a different AWS Region.

Question 3

A data scientist is using Amazon SageMaker to train a deep learning model with a large dataset. The training job fails with a 'CUDA out of memory' error. What is the MOST efficient way to resolve this issue?

Accepted Answer

Reduce the batch size. The 'CUDA out of memory' error occurs when the GPU's memory is insufficient to hold the model parameters, gradients, optimizer states, and the current batch of data. Reducing the batch size decreases the memory footprint per training step, allowing the model to fit within the available GPU memory without requiring a more expensive instance or sacrificing GPU acceleration.

Answer

Switch to a CPU-only instance

Answer

Use a larger instance type with more GPUs

Answer

Increase the batch size

Question 4

A data scientist is using Amazon SageMaker to train a neural network. The training job fails with the error 'ResourceLimitExceeded: The account-level service limit for ml.p3.8xlarge for training job usage is 0.' What is the most likely cause and solution?

Accepted Answer

The account has not requested a limit increase for ml.p3.8xlarge; submit a limit increase request via AWS Support.. The error message explicitly states that the account-level service limit for ml.p3.8xlarge for training job usage is 0, which means the account has not been granted any capacity for that instance type. AWS enforces service quotas (limits) per account per region, and for GPU-intensive instances like ml.p3.8xlarge, the default limit is often 0 unless a limit increase request has been submitted and approved. Therefore, the correct solution is to request a limit increase via AWS Support.

Answer

The training job is using spot instances; switch to on-demand instances.

Answer

The instance type is not available in the current region; switch to a different region.

Answer

The instance type is too large; use a smaller instance type like ml.m5.large.

Question 5

A data scientist is troubleshooting a failed SageMaker training job that uses a custom Docker image. The failure reason shows 'unrecognized arguments: --sagemaker_program'. What is the most likely cause?

Accepted Answer

The custom Docker image does not use the SageMaker training toolkit and thus does not accept SageMaker hyperparameters. The error 'unrecognized arguments: --sagemaker_program' indicates that the custom Docker image does not include the SageMaker Training Toolkit. The SageMaker Training Toolkit is a Python library that provides a default entry point to parse and handle SageMaker-specific hyperparameters (like --sagemaker_program, --sagemaker_submit_directory, etc.). Without this toolkit, the container's entry point does not recognize these arguments, causing the training job to fail.

Answer

The Docker image is tagged incorrectly and cannot be pulled

Answer

The training job is in a different region than the ECR repository

Answer

The input mode is File mode, but the container expects Pipe mode

Question 6

A data scientist is analyzing a dataset containing customer reviews. The data scientist wants to understand the most common words used in positive and negative reviews. Which AWS service is most suitable for this task?

Accepted Answer

Amazon Comprehend. Amazon Comprehend is a fully managed NLP service that provides entity recognition, sentiment analysis, key phrase extraction, and topic modeling — exactly what is needed to identify the most common words/phrases in positive versus negative reviews. It handles tokenization, stop-word removal, and language detection out of the box, so the data scientist does not need to build custom NLP pipelines. Comprehend's 'DetectKeyPhrases' and 'DetectSentiment' APIs can be combined to bucket reviews by sentiment and then surface dominant terms.

Answer

Amazon Rekognition

Answer

Amazon Polly

Answer

Amazon Transcribe

Question 7

A data scientist is training a deep learning model on a large dataset using Amazon SageMaker. The training job is taking too long. Which action would MOST likely reduce training time without sacrificing model accuracy?

Accepted Answer

Enable data parallelism across multiple GPUs. Enabling data parallelism across multiple GPUs distributes the training workload across several devices, allowing larger batch sizes and faster gradient computation per epoch. Amazon SageMaker's distributed training libraries (e.g., SageMaker Data Parallelism) use all-reduce algorithms to synchronize gradients efficiently, which reduces wall-clock training time without altering the model architecture or loss function, thus preserving accuracy.

Answer

Use a smaller instance type for training

Answer

Implement early stopping with a low patience value

Answer

Reduce the number of epochs by half

Question 8

A data scientist is training a neural network for a multi-class classification problem with 100 classes. The model uses a softmax output layer and cross-entropy loss. During training, the loss decreases steadily but the accuracy on the validation set plateaus early. Which of the following is the most likely cause?

Accepted Answer

The model is overfitting the training data. When the validation accuracy plateaus early while training loss continues to decrease, it indicates that the model is memorizing the training data rather than learning generalizable patterns. This is classic overfitting, where the softmax output layer produces high-confidence predictions for training samples but fails to generalize to unseen validation data, causing cross-entropy loss to drop on the training set while validation accuracy stagnates.

Answer

Batch size is too large

Answer

Number of epochs is too small

Answer

Learning rate is too high

Question 9

A company is using Amazon SageMaker to train a deep learning model for image segmentation. The training job uses a single ml.p3.2xlarge instance and takes 48 hours to complete. The team needs to reduce training time to under 12 hours to meet a deadline. The dataset is 50 GB of images stored in S3. The team currently uses File mode to download the data to the training instance. The model architecture is a convolutional neural network (CNN) with 50 layers. The team has access to multiple instances of the same type. Which approach will most effectively reduce training time?

Accepted Answer

Use SageMaker's distributed data parallelism with multiple instances.. SageMaker's distributed data parallelism splits the 50 GB dataset across multiple ml.p3.2xlarge instances, allowing each instance to process a subset of the data in parallel. This can reduce training time from 48 hours to under 12 hours, assuming near-linear scaling with the number of instances (e.g., 4 instances for a 4x speedup). The approach directly addresses the need to reduce wall-clock time without altering the model architecture or data loading method.

Answer

Reduce the number of layers in the CNN to speed up training.

Answer

Increase the batch size on the single instance to process more data per iteration.

Answer

Switch to Pipe mode to stream data from S3, reducing data loading time.

Question 10

A machine learning engineer is evaluating a binary classification model. The model has a high recall but low precision. Which of the following is the most likely consequence?

Accepted Answer

The model has many false positives.. High recall means the model correctly identifies most positive cases (few false negatives), but low precision indicates that among the cases predicted as positive, many are actually negative. This directly implies a high number of false positives, as precision = TP/(TP+FP) and a low precision with high recall forces FP to be large relative to TP.

Answer

The model has few false negatives.

Answer

The model misses many positive cases.

Answer

The model has few false positives.

Question 11

A team trained a multiclass classification model using SageMaker built-in XGBoost. The model's accuracy is high, but for a specific class, recall is very low. The team wants to improve recall for that class without significant accuracy drop. Which approach is MOST effective?

Accepted Answer

Resample the training data to balance the class representation. Resampling the training data to balance class representation directly addresses the root cause of low recall for a specific class in a multiclass XGBoost model. XGBoost's built-in objective functions (e.g., 'multi:softmax') optimize for overall accuracy, which can bias the model toward majority classes; resampling (e.g., oversampling the minority class or undersampling the majority) forces the model to learn decision boundaries that better capture the minority class, improving recall without drastically reducing overall accuracy.

Answer

Add more training data from all classes

Answer

Increase the max_depth hyperparameter of XGBoost

Answer

Switch from XGBoost to a linear learner

Question 12

A data scientist is using Amazon SageMaker to train a custom TensorFlow model. The training job is failing with the error: 'OutOfRangeError: End of sequence'. The input data is stored in TFRecord format in S3. What is the most likely cause?

Accepted Answer

The number of training steps or epochs specified exceeds the dataset size.. The 'OutOfRangeError: End of sequence' error in TensorFlow occurs when the training loop attempts to read more data than is available in the dataset. This typically happens when the number of training steps or epochs specified exceeds the total number of records in the TFRecord files, causing the iterator to reach the end of the dataset prematurely.

Answer

The TFRecord files are corrupted.

Answer

The instance type does not have enough memory.

Answer

The shuffle buffer size is too large.

Question 13

During training of a SageMaker built-in object detection algorithm, the loss is not decreasing after several epochs. Which troubleshooting step should be taken first?

Accepted Answer

Check whether the learning rate is appropriate. When the loss is not decreasing during training of a SageMaker built-in object detection algorithm, the most common cause is an inappropriate learning rate. A learning rate that is too high can cause the loss to oscillate or diverge, while one that is too low can cause the loss to plateau. Checking and adjusting the learning rate is the first troubleshooting step because it directly controls the step size of gradient updates and is a fundamental hyperparameter in optimization.

Answer

Increase the mini-batch size

Answer

Add more classes to the dataset

Answer

Increase the number of epochs

Question 14

A machine learning engineer is building a binary classification model to predict customer churn. The dataset is highly imbalanced (5% churn). The engineer wants to use Amazon SageMaker's built-in XGBoost algorithm. Which combination of hyperparameters is most appropriate for this scenario?

Accepted Answer

scale_pos_weight=19, subsample=0.8. In a highly imbalanced dataset with only 5% churn, the ratio of negative to positive classes is 95:5, or 19:1. The `scale_pos_weight` hyperparameter in XGBoost should be set to this ratio (19) to penalize misclassifications of the minority class more heavily. A `subsample` of 0.8 introduces stochasticity and helps prevent overfitting, which is especially important when the minority class is small.

Answer

scale_pos_weight=0.05, subsample=0.8

Answer

scale_pos_weight=19, subsample=1.0

Answer

scale_pos_weight=1, subsample=1.0

Question 15

A company is building a text classification model to categorize customer support tickets. The dataset is highly imbalanced with 95% of tickets belonging to 'General Inquiry' and 5% to 'Complaint'. The data scientist is using a random forest classifier. Which metric is most appropriate for evaluating model performance on the minority class?

Accepted Answer

F1-score for 'Complaint'. In a highly imbalanced dataset (95% General Inquiry, 5% Complaint), accuracy is misleading because a model that predicts 'General Inquiry' for every ticket would achieve 95% accuracy but completely fail on the minority class. The F1-score for 'Complaint' is the harmonic mean of precision and recall, providing a balanced evaluation of the model's ability to correctly identify complaints without being skewed by the majority class. For a random forest classifier, this metric directly addresses the minority class performance, which is the primary concern.

Answer

Accuracy

Answer

ROC AUC

Answer

Precision for 'Complaint'

Question 16

A data scientist is using SageMaker to train a linear learner algorithm. After training, the evaluation shows that the model has high bias. Which action is most likely to reduce bias?

Accepted Answer

Add feature crosses for categorical variables. High bias indicates that the model is underfitting the data, meaning it is too simple to capture the underlying patterns. Adding feature crosses for categorical variables creates interaction features that allow the linear learner to model non-linear relationships, increasing model complexity and reducing bias. This is a standard technique in linear models to address underfitting without switching to a non-linear algorithm.

Answer

Increase the L2 regularization strength

Answer

Reduce the amount of training data

Answer

Remove some features that have low variance

Question 17

A company is using Amazon SageMaker to train a linear regression model. The data scientist notices that the training loss is decreasing but the validation loss has started to increase after a few epochs. What is the most likely cause?

Accepted Answer

The model is overfitting the training data.. A diverging pattern — training loss continuing to fall while validation loss rises — is the textbook signature of overfitting: the model is memorizing training-specific noise rather than learning generalizable patterns. In SageMaker linear regression this typically appears after several epochs once the model has enough capacity relative to the data.

Answer

The model is underfitting the training data.

Answer

There is data leakage from the validation set into the training set.

Answer

The learning rate is too high.

Question 18

A data scientist is training a binary classification model on a highly imbalanced dataset (99% negative class, 1% positive class). The model currently achieves 99% accuracy but only identifies 0.5% of true positives. Which metric should the data scientist focus on to improve model performance?

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual positives correctly identified, which is critical when the dataset is highly imbalanced (99% negative, 1% positive) and the model fails to detect most positives (only 0.5% true positives). Improving recall directly addresses the model's inability to capture the minority class, even if it reduces precision or accuracy. In binary classification with severe class imbalance, accuracy is misleading because a model can achieve 99% accuracy by simply predicting the majority class, as seen here.

Answer

Precision

Answer

Root Mean Squared Error (RMSE)

Answer

Accuracy

Question 19

A data engineer needs to schedule an AWS Glue ETL job to run every hour. Which service should they use for scheduling?

Accepted Answer

Amazon CloudWatch Events. Amazon CloudWatch Events (now part of Amazon EventBridge) can trigger an AWS Glue ETL job on a schedule using a cron or rate expression. This is the native, serverless scheduling service for running jobs at fixed intervals, such as every hour, without needing to manage any infrastructure.

Answer

AWS Lambda

Answer

Amazon Simple Queue Service (SQS)

Answer

AWS Step Functions

Question 20

A retail company uses Amazon SageMaker to train a model for product demand forecasting. The dataset contains daily sales data for 10,000 products over 3 years. The data includes features like price, promotions, holidays, and seasonality. The data scientist uses a linear regression model and gets an RMSE of 50 units. However, the business requires more accurate forecasts, especially for products with high variability. The scientist notices that the residuals show a pattern: the model underestimates demand during promotional periods. Which approach should the scientist take to improve the model?

Accepted Answer

Add interaction features between promotion and other variables.. The residuals show a systematic pattern where the model underestimates demand during promotions, indicating the linear model cannot capture the interaction between promotions and other features. Adding interaction features (e.g., promotion × price, promotion × holiday) allows the linear model to represent these conditional relationships, improving accuracy for high-variability products. This is a targeted feature engineering fix that directly addresses the observed bias.

Answer

Collect more historical data for training.

Answer

Use a deep learning model like LSTM.

Answer

Remove promotion features to simplify the model.