MLS-C01 Mock Exam 91 — 25 Questions

Question 1

A company is building a recommendation system using Amazon SageMaker's Factorization Machines algorithm. The dataset includes user IDs, item IDs, and ratings. The data is sparse. Which data format should be used for training?

Accepted Answer

RecordIO-protobuf format with sparse feature vectors.. Option C is correct because Factorization Machines (FM) in SageMaker are optimized for sparse, high-dimensional data. The RecordIO-protobuf format allows you to directly specify sparse feature vectors using integer keys and float values, which avoids the memory overhead of dense representations and enables efficient distributed training. This format is the recommended input for SageMaker's built-in FM algorithm.

Answer

CSV format with one row per rating.

Answer

JSON lines format with nested structures.

Answer

Parquet format with columns for each feature.

Question 2

A company is using SageMaker Autopilot to automatically build a binary classification model. After the AutoML job completes, the data scientist wants to understand which features are most important for the best candidate model. How can the scientist get feature importance?

Accepted Answer

Open the SageMaker Autopilot job details and view the 'Explainability' tab. SageMaker Autopilot automatically generates a 'Explainability' tab within the job details for the best candidate model. This tab uses SHAP (SHapley Additive exPlanations) values to provide feature importance, showing which features most influence the model's predictions. The data scientist can directly access this information without any additional configuration or re-running the model.

Answer

Re-run the best model using SageMaker built-in XGBoost with the 'feature_importance' hyperparameter

Answer

Check the CloudWatch Logs for the training job

Answer

Use SageMaker Ground Truth to label a new dataset

Question 3

A data scientist is using Amazon SageMaker to train a linear regression model. The training data has 10 features and 100,000 observations. The model's training loss is decreasing, but the validation loss starts increasing after a few epochs. Which step should the data scientist take first to address this issue?

Accepted Answer

Reduce the learning rate. The increasing validation loss while training loss decreases is a classic sign of overfitting. Reducing the learning rate (Option B) is the first step to stabilize training by allowing the optimizer to take smaller, more controlled steps, which can help the model converge to a better local minimum and reduce validation loss. In SageMaker, this is typically adjusted via the `learning_rate` hyperparameter in the estimator.

Answer

Add more features to the model

Answer

Increase the batch size

Answer

Increase the number of epochs

Question 4

A data scientist is trying to launch a SageMaker training job using an IAM role with the above policy. The training job fails with an access denied error. What is the MOST likely reason?

Accepted Answer

The policy does not include s3:ListBucket permission. The policy grants s3:GetObject and s3:PutObject permissions on the S3 bucket ARN with a /* suffix, but SageMaker training jobs also require s3:ListBucket permission at the bucket level (without the /*) to enumerate objects and validate paths during job creation. Without s3:ListBucket, the training job fails with an access denied error even though read/write permissions are present.

Answer

The sagemaker:StopTrainingJob action is not required

Answer

The sagemaker:CreateTrainingJob action is not allowed for the specific instance type

Answer

The S3 bucket ARN should not include the /* suffix

Question 5

A data scientist is training a binary classifier on an imbalanced dataset where the positive class represents 1% of the data. The model is evaluated using accuracy, but the accuracy is 99% even though the model predicts all instances as negative. Which metric should the data scientist use to properly evaluate the model?

Accepted Answer

F1 score. The F1 score is the harmonic mean of precision and recall, making it robust to class imbalance. With 99% negative instances, accuracy is misleadingly high even if the model never predicts the positive class. F1 captures both false positives and false negatives, providing a balanced evaluation of the minority class performance.

Answer

Root mean squared error (RMSE)

Answer

Mean squared error (MSE)

Answer

Accuracy

Question 6

A company uses Amazon SageMaker to deploy a model that predicts customer churn. The model is retrained weekly. The data scientist notices that the model's accuracy remains high, but the business reports that the model is not capturing new churn patterns. What is the most likely cause?

Accepted Answer

The model is experiencing concept drift. Concept drift occurs when the statistical properties of the target variable change over time, causing the model's predictions to become less relevant even if accuracy metrics remain high. In this scenario, the model is retrained weekly but still fails to capture new churn patterns because the underlying customer behavior has shifted—a classic sign of concept drift rather than a data or overfitting issue. Amazon SageMaker's built-in Model Monitor can detect such drift by comparing inference data distributions against a baseline.

Answer

The model is underfitting the data

Answer

The model has data leakage from future data

Answer

The model is overfitting to the training data

Question 7

A data scientist is building a text classification model using a bag-of-words approach with logistic regression. The dataset has 10,000 documents and 50,000 unique tokens. The model overfits. Which TWO techniques can help reduce overfitting?

Accepted Answer

Reduce the vocabulary size by removing rare and very frequent terms. Option D is correct because removing rare and very frequent terms reduces the feature space and eliminates noise, which helps the logistic regression model generalize better. Rare terms often act as noise that the model can latch onto for spurious correlations, while very frequent terms (like stopwords) provide little discriminative power. This dimensionality reduction directly combats overfitting by simplifying the model.

Answer

Increase the number of n-grams features

Answer

Use one-hot encoding instead of bag-of-words

Answer

Use a more complex model such as a neural network

Question 8

A company is deploying a real-time inference endpoint using SageMaker. The model is a large deep learning model (5 GB) with strict latency requirements (< 100 ms per request). The team expects bursty traffic with up to 1000 requests per second. Which configuration best meets the latency and throughput requirements?

Accepted Answer

Deploy an ml.p3.2xlarge instance with automatic scaling based on a custom metric like 'InvocationsPerInstance'. Option A is correct because deploying on an ml.p3.2xlarge instance with automatic scaling based on 'InvocationsPerInstance' allows the endpoint to handle bursty traffic up to 1000 requests per second while maintaining sub-100 ms latency. The GPU-accelerated p3 instance provides the necessary compute for a 5 GB deep learning model, and custom scaling on invocations per instance ensures that additional instances are provisioned quickly during traffic spikes without over-provisioning.

Answer

Use a multi-model endpoint with ml.c5.4xlarge instances

Answer

Use SageMaker Serverless Inference with a memory size of 6 GB

Answer

Deploy a single ml.p3.16xlarge instance with a production variant

Question 9

An IAM policy attached to a SageMaker execution role is shown. A training job executed with this role fails with an error that the role cannot access the S3 bucket. The training job uses input data from s3://my-bucket/train/data.csv and output to s3://my-bucket/output/. What is the most likely cause?

Accepted Answer

The training job does not have s3:PutObject permission for the output location. Option C is correct because the error message indicates the role cannot access the S3 bucket, which typically occurs when the role lacks write permissions to the output location. The training job needs s3:PutObject permission to write the output artifacts (model, logs, etc.) to s3://my-bucket/output/. Without this permission, SageMaker fails to save the training results, resulting in an access error.

Answer

The training job does not have s3:GetObject permission for the input data

Answer

The training data is encrypted with SSE-KMS and the role lacks KMS permissions

Answer

The S3 bucket is in a different region than the training job

Question 10

A data scientist is using SageMaker to train a linear learner algorithm. After training, the evaluation shows that the model has high bias. Which action is most likely to reduce bias?

Accepted Answer

Add feature crosses for categorical variables. High bias indicates that the model is underfitting the data, meaning it is too simple to capture the underlying patterns. Adding feature crosses for categorical variables creates interaction features that allow the linear learner to model non-linear relationships, increasing model complexity and reducing bias. This is a standard technique in linear models to address underfitting without switching to a non-linear algorithm.

Answer

Increase the L2 regularization strength

Answer

Reduce the amount of training data

Answer

Remove some features that have low variance

Question 11

During training of a SageMaker built-in object detection algorithm, the loss is not decreasing after several epochs. Which troubleshooting step should be taken first?

Accepted Answer

Check whether the learning rate is appropriate. When the loss is not decreasing during training of a SageMaker built-in object detection algorithm, the most common cause is an inappropriate learning rate. A learning rate that is too high can cause the loss to oscillate or diverge, while one that is too low can cause the loss to plateau. Checking and adjusting the learning rate is the first troubleshooting step because it directly controls the step size of gradient updates and is a fundamental hyperparameter in optimization.

Answer

Increase the mini-batch size

Answer

Add more classes to the dataset

Answer

Increase the number of epochs

Question 12

A data scientist is using Amazon SageMaker to train a custom TensorFlow model. The training job is failing with the error: 'OutOfRangeError: End of sequence'. The input data is stored in TFRecord format in S3. What is the most likely cause?

Accepted Answer

The number of training steps or epochs specified exceeds the dataset size.. The 'OutOfRangeError: End of sequence' error in TensorFlow occurs when the training loop attempts to read more data than is available in the dataset. This typically happens when the number of training steps or epochs specified exceeds the total number of records in the TFRecord files, causing the iterator to reach the end of the dataset prematurely.

Answer

The TFRecord files are corrupted.

Answer

The instance type does not have enough memory.

Answer

The shuffle buffer size is too large.

Question 13

A data scientist is reviewing the training logs from a SageMaker training job. The logs show training and validation loss per epoch. Based on the exhibited logs, which statement is correct?

Accepted Answer

The model is overfitting because training loss decreases while validation loss does not. Option D is correct because the training logs show a classic overfitting pattern: training loss consistently decreases across epochs, indicating the model is memorizing the training data, while validation loss does not decrease (or may even increase), indicating poor generalization to unseen data. In SageMaker, monitoring both losses during training is critical to detect overfitting early, often prompting regularization or early stopping.

Answer

The model is not learning because the loss is not decreasing

Answer

The model is underfitting because both losses are high

Answer

The model is performing well because validation loss is stable

Question 14

A data scientist is building a regression model to predict house prices. The dataset contains features like 'number_of_rooms' (integer), 'sqft' (float), 'location' (categorical with 1000 unique values). Which feature engineering approach is BEST for the 'location' feature?

Accepted Answer

Target encoding. Target encoding is the best approach for the 'location' feature because it has 1,000 unique categories, making one-hot encoding infeasible (would create 1,000 dummy columns) and label encoding inappropriate (imposes arbitrary ordinal relationships). Target encoding replaces each category with the mean of the target variable (house price) for that category, capturing the predictive signal of location while keeping the feature as a single numeric column. This balances model performance with dimensionality and avoids overfitting when regularized (e.g., with smoothing or cross-validation).

Answer

Remove the feature

Answer

One-hot encoding

Answer

Label encoding

Question 15

A machine learning engineer is building a binary classification model to predict customer churn. The dataset is highly imbalanced (5% churn). The engineer wants to use Amazon SageMaker's built-in XGBoost algorithm. Which combination of hyperparameters is most appropriate for this scenario?

Accepted Answer

scale_pos_weight=19, subsample=0.8. In a highly imbalanced dataset with only 5% churn, the ratio of negative to positive classes is 95:5, or 19:1. The `scale_pos_weight` hyperparameter in XGBoost should be set to this ratio (19) to penalize misclassifications of the minority class more heavily. A `subsample` of 0.8 introduces stochasticity and helps prevent overfitting, which is especially important when the minority class is small.

Answer

scale_pos_weight=0.05, subsample=0.8

Answer

scale_pos_weight=19, subsample=1.0

Answer

scale_pos_weight=1, subsample=1.0

Question 16

A machine learning team is deploying a time-series forecasting model using Amazon SageMaker. The model is trained on historical data and needs to be updated daily with new data. The team wants to automate the retraining pipeline and avoid manual intervention. Which approach is the most efficient?

Accepted Answer

Use Amazon SageMaker Pipelines with a scheduled Lambda function to trigger retraining daily.. Option D is correct because Amazon SageMaker Pipelines provides a fully managed, end-to-end orchestration service for building, training, and deploying machine learning models. By combining it with a scheduled AWS Lambda function, the team can automate daily retraining without manual intervention, leveraging SageMaker's native integration for step sequencing, artifact tracking, and model registry updates.

Answer

Use AWS Step Functions to orchestrate retraining, but require a manual approval step.

Answer

Use SageMaker training jobs manually triggered by the team each day.

Answer

Use a cron job on an EC2 instance to run a training script.

Question 17

A team is using Amazon SageMaker to train a model. The training job repeatedly fails with a 'ResourceLimitExceeded' error. Which action should the team take to resolve this issue?

Accepted Answer

Request a service limit increase for SageMaker resources.. The 'ResourceLimitExceeded' error in Amazon SageMaker indicates that the AWS account has reached a service quota for SageMaker resources, such as the number of concurrent training jobs, total instance count, or specific instance types. The correct action is to request a service limit increase via the AWS Service Quotas console or by contacting AWS Support, as this directly addresses the quota cap causing the failure.

Answer

Reduce the size of the training dataset.

Answer

Switch to using Spot Instances.

Answer

Use a different instance type with less memory.

Question 18

A company is building a recommendation system for an e-commerce platform. The system needs to suggest products to users based on past purchases and browsing history. Which approach would be most appropriate for this use case?

Accepted Answer

Collaborative filtering using past user-item interactions. Collaborative filtering is the most appropriate approach because it leverages past user-item interactions (e.g., purchases, clicks) to identify patterns and recommend items that similar users have liked. This method directly captures user behavior and preferences without requiring explicit product metadata, making it ideal for e-commerce recommendation systems where implicit feedback is abundant.

Answer

Content-based filtering using product descriptions

Answer

K-means clustering of users based on demographics

Answer

Matrix factorization on user-item ratings

Question 19

A machine learning team is deploying a model for real-time fraud detection. The model must make predictions with less than 100ms latency. The team uses SageMaker and the model is a large ensemble of decision trees. Which SageMaker hosting option is MOST suitable?

Accepted Answer

SageMaker Multi-model endpoint. A SageMaker Multi-model endpoint is the most suitable option because it allows hosting multiple models (including large ensembles) on a single endpoint while sharing underlying compute resources, which reduces cost and latency. The endpoint dynamically loads and caches models based on inference requests, enabling sub-100ms predictions for large ensemble models by keeping frequently used models in memory.

Answer

SageMaker Serverless Inference

Answer

SageMaker Elastic Inference

Answer

SageMaker Batch Transform

Question 20

A machine learning engineer is deploying a model that predicts customer churn. The model outputs probabilities between 0 and 1. The business requires that at least 90% of customers flagged for churn actually churn (precision >= 0.9). Currently, the model's precision is 0.85 at the default threshold of 0.5. Which threshold adjustment should the engineer consider?

Accepted Answer

Increase the threshold to 0.7. Increasing the threshold to 0.7 raises the probability cutoff for classifying a customer as churning. This means only customers with a high predicted probability (strong model confidence) are flagged, which reduces false positives and increases precision. Since the current precision at 0.5 is 0.85 and the goal is ≥0.9, moving the threshold higher is the correct direction to achieve the required precision.

Answer

Decrease the threshold to 0.4

Answer

Decrease the threshold to 0.3

Answer

Keep the threshold at 0.5