MLS-C01 Modeling Practice Test 37 — 15 Questions

Question 1

A company is building a recommendation system using Amazon SageMaker's Factorization Machines algorithm. The dataset includes user IDs, item IDs, and ratings. The data is sparse. Which data format should be used for training?

Accepted Answer

RecordIO-protobuf format with sparse feature vectors.. Option C is correct because Factorization Machines (FM) in SageMaker are optimized for sparse, high-dimensional data. The RecordIO-protobuf format allows you to directly specify sparse feature vectors using integer keys and float values, which avoids the memory overhead of dense representations and enables efficient distributed training. This format is the recommended input for SageMaker's built-in FM algorithm.

Answer

CSV format with one row per rating.

Answer

JSON lines format with nested structures.

Answer

Parquet format with columns for each feature.

Question 2

A data scientist needs to evaluate a binary classification model. The dataset is balanced. Which metric is most appropriate to compare model performance?

Accepted Answer

Accuracy. For a balanced binary classification dataset, accuracy is the most appropriate metric because it directly measures the proportion of correct predictions (true positives and true negatives) out of all predictions. Since the class distribution is equal, accuracy is not misleadingly high due to class imbalance, making it a reliable and straightforward measure of overall model performance.

Answer

Recall

Answer

F1 score

Answer

Precision

Question 3

A company uses Amazon SageMaker to train a model. The training job runs successfully but the model artifacts are not saved to the specified S3 output path. What is a likely cause?

Accepted Answer

The training script does not save the model to /opt/ml/model.. Option A is correct because Amazon SageMaker expects the training script to save the model artifacts to the `/opt/ml/model` directory. After the training job completes, SageMaker automatically copies the contents of this directory to the specified S3 output path. If the script saves the model elsewhere (e.g., `/tmp` or a custom path), no artifacts will be uploaded, resulting in an empty or missing S3 output.

Answer

The model size exceeds the S3 bucket limit.

Answer

The training job used spot instances.

Answer

The S3 bucket is in a different AWS Region.

Question 4

A data scientist is training a deep learning model on Amazon SageMaker and wants to reduce the training time. Which TWO actions would help achieve this?

Accepted Answer

Use distributed training across multiple instances.. Distributed training across multiple instances (Option B) reduces training time by parallelizing the workload across multiple compute nodes, leveraging data parallelism or model parallelism to process larger batches or model partitions simultaneously. This is particularly effective for deep learning models where the dataset or model size exceeds the capacity of a single instance, as it scales throughput linearly with the number of instances under ideal conditions.

Answer

Enable data augmentation.

Answer

Use SageMaker Automatic Model Tuning.

Answer

Use SageMaker Managed Spot Training.

Question 5

A company is deploying a machine learning model for fraud detection. The model outputs a probability score. The cost of false negatives is very high. Which TWO metrics should the company focus on optimizing?

Accepted Answer

F1 score. Recall (true positive rate) measures ability to find positives; minimizing false negatives is optimizing recall. AUC-ROC summarizes overall performance but not specific to false negatives. Precision focuses on false positives. FPR is about false positives. F1 balances precision and recall, but recall directly addresses false negatives.

Answer

Precision

Answer

False positive rate (FPR)

Answer

Area under the ROC curve (AUC-ROC)

Question 6

A data scientist is using Amazon SageMaker to train a linear regression model. The training data contains outliers. Which THREE techniques can mitigate the impact of outliers?

Accepted Answer

Remove observations with outlier values from the dataset.. Option A is correct because removing observations with outlier values directly eliminates data points that can disproportionately influence the linear regression coefficients, leading to a more stable and representative model. In Amazon SageMaker, this can be done during data preprocessing using built-in algorithms or custom scripts in a SageMaker Processing job.

Answer

Increase the number of layers in the model.

Answer

Standardize the features to have mean zero and unit variance.

Question 7

A data scientist is training a binary classifier on an imbalanced dataset where the positive class represents 1% of the data. The model is evaluated using accuracy, but the accuracy is 99% even though the model predicts all instances as negative. Which metric should the data scientist use to properly evaluate the model?

Accepted Answer

F1 score. The F1 score is the harmonic mean of precision and recall, making it robust to class imbalance. With 99% negative instances, accuracy is misleadingly high even if the model never predicts the positive class. F1 captures both false positives and false negatives, providing a balanced evaluation of the minority class performance.

Answer

Root mean squared error (RMSE)

Answer

Mean squared error (MSE)

Answer

Accuracy

Question 8

A machine learning team is deploying a time-series forecasting model using Amazon SageMaker. The model is trained on historical data and needs to be updated daily with new data. The team wants to automate the retraining pipeline and avoid manual intervention. Which approach is the most efficient?

Accepted Answer

Use Amazon SageMaker Pipelines with a scheduled Lambda function to trigger retraining daily.. Option D is correct because Amazon SageMaker Pipelines provides a fully managed, end-to-end orchestration service for building, training, and deploying machine learning models. By combining it with a scheduled AWS Lambda function, the team can automate daily retraining without manual intervention, leveraging SageMaker's native integration for step sequencing, artifact tracking, and model registry updates.

Answer

Use AWS Step Functions to orchestrate retraining, but require a manual approval step.

Answer

Use SageMaker training jobs manually triggered by the team each day.

Answer

Use a cron job on an EC2 instance to run a training script.

Question 9

A data scientist is training a linear regression model. After training, the model has a high bias and low variance. Which technique should the data scientist use to reduce bias?

Accepted Answer

Add more relevant features. High bias indicates the model is underfitting the data, meaning it is too simple to capture the underlying patterns. Adding more relevant features increases model complexity, allowing it to learn more from the data and reduce bias. This directly addresses the underfitting issue without increasing variance excessively, provided the features are meaningful.

Answer

Decrease the model complexity

Answer

Apply L2 regularization (Ridge)

Answer

Reduce the amount of training data

Question 10

A company is building a recommendation system for an e-commerce platform. The system needs to suggest products to users based on past purchases and browsing history. Which approach would be most appropriate for this use case?

Accepted Answer

Collaborative filtering using past user-item interactions. Collaborative filtering is the most appropriate approach because it leverages past user-item interactions (e.g., purchases, clicks) to identify patterns and recommend items that similar users have liked. This method directly captures user behavior and preferences without requiring explicit product metadata, making it ideal for e-commerce recommendation systems where implicit feedback is abundant.

Answer

Content-based filtering using product descriptions

Answer

K-means clustering of users based on demographics

Answer

Matrix factorization on user-item ratings

Question 11

A data scientist is training a deep learning model for image classification. The model is overfitting on the training data. Which combination of techniques will most effectively reduce overfitting?

Accepted Answer

Add dropout layers and use data augmentation. Dropout layers randomly deactivate a fraction of neurons during training, which forces the network to learn more robust features and prevents co-adaptation. Data augmentation artificially expands the training dataset by applying transformations (e.g., rotation, flipping, cropping), which reduces the model's ability to memorize spurious patterns and improves generalization. Together, these techniques directly counteract overfitting by increasing regularization and effective training diversity.

Answer

Reduce the batch size

Answer

Train for more epochs without early stopping

Answer

Increase the number of layers and neurons

Question 12

A machine learning engineer is evaluating a binary classification model. The model has a high recall but low precision. Which of the following is the most likely consequence?

Accepted Answer

The model has many false positives.. High recall means the model correctly identifies most positive cases (few false negatives), but low precision indicates that among the cases predicted as positive, many are actually negative. This directly implies a high number of false positives, as precision = TP/(TP+FP) and a low precision with high recall forces FP to be large relative to TP.

Answer

The model has few false negatives.

Answer

The model misses many positive cases.

Answer

The model has few false positives.

Question 13

A data scientist is working on a regression problem with a dataset that contains outliers. The data scientist is choosing between mean squared error (MSE) and mean absolute error (MAE) as the loss function. Which loss function is more robust to outliers?

Accepted Answer

MAE, because it treats all errors equally.. MAE is more robust to outliers because it uses the absolute difference between predicted and actual values, which does not disproportionately penalize large errors. In contrast, MSE squares the errors, causing outliers to have a much larger influence on the loss and model updates. This makes MAE less sensitive to extreme values in regression tasks.

Answer

Both are equally robust to outliers.

Answer

MSE, because it penalizes large errors more heavily.

Answer

Neither is robust; use Huber loss instead.

Question 14

A team is using Amazon SageMaker to train a model. The training job repeatedly fails with a 'ResourceLimitExceeded' error. Which action should the team take to resolve this issue?

Accepted Answer

Request a service limit increase for SageMaker resources.. The 'ResourceLimitExceeded' error in Amazon SageMaker indicates that the AWS account has reached a service quota for SageMaker resources, such as the number of concurrent training jobs, total instance count, or specific instance types. The correct action is to request a service limit increase via the AWS Service Quotas console or by contacting AWS Support, as this directly addresses the quota cap causing the failure.

Answer

Reduce the size of the training dataset.

Answer

Switch to using Spot Instances.

Answer

Use a different instance type with less memory.

Question 15

A data scientist is using a decision tree algorithm for a classification task. The tree is very deep and achieves 100% accuracy on the training set but performs poorly on the test set. Which technique should the data scientist use to improve generalization?

Accepted Answer

Prune the decision tree.. A deep decision tree that achieves 100% training accuracy but poor test accuracy is overfitting the training data. Pruning the tree removes branches that have little statistical power, reducing complexity and improving generalization to unseen data.

Answer

Add more features to the dataset.

Answer

Reduce the number of training samples.

Answer

Increase the maximum depth of the tree.