MLS-C01 Modeling Practice Test 13 — 15 Questions

Question 1

A data scientist is evaluating a binary classification model that predicts whether a customer will churn. The model achieves an AUC of 0.85 on the test set. Which TWO statements about AUC are correct? (Choose two.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

AUC represents the probability that a randomly chosen positive instance is ranked higher than a randomly chosen negative instance.. AUC measures the probability that a randomly chosen positive instance is ranked higher than a randomly chosen negative instance, which is correctly stated in option A. AUC is also threshold-independent, evaluating model ranking across all thresholds, as stated in option E. Option B is incorrect because an AUC of 0.85 is better than random (0.5). Option C is incorrect because AUC is not average precision; average precision is a different metric. Option D is incorrect because AUC is not equivalent to accuracy at any specific threshold.

Answer

An AUC of 0.85 indicates the model is no better than random guessing.

Answer

AUC is the average precision across all thresholds.

Answer

AUC is equivalent to the accuracy of the model at the default threshold of 0.5.

Question 2

A data scientist trains a linear regression model to predict house prices. The model has high bias (underfitting). Which action is most likely to reduce bias?

Accepted Answer

Increase model complexity. Increasing model complexity (e.g., adding polynomial features or using a more flexible algorithm) can reduce bias. Adding L1 regularization increases bias, reducing features reduces complexity, and lowering max_depth for a tree also increases bias.

Answer

Reduce the number of features

Answer

Decrease the maximum depth of the tree

Answer

Add L1 regularization

Question 3

A company is building a binary classifier to detect fraudulent transactions. The dataset is highly imbalanced (99% legitimate, 1% fraudulent). Which metric is most appropriate for evaluating the model?

Accepted Answer

F1-score. Precision and recall (or F1-score) are more informative for imbalanced datasets than accuracy, because a model predicting all legitimate would achieve 99% accuracy but be useless. F1-score balances precision and recall.

Answer

Accuracy

Answer

Mean Squared Error

Answer

Area Under the ROC Curve (AUC-ROC)

Question 4

A machine learning team trains a deep learning model on SageMaker. The training job uses a single ml.p3.2xlarge instance and takes 12 hours. The team needs to reduce training time without changing the algorithm. Which approach is most effective?

Accepted Answer

Use SageMaker's distributed data parallelism across multiple instances. Using SageMaker's distributed data parallelism (e.g., with SageMaker distributed training libraries) across multiple GPUs can significantly reduce training time by splitting the mini-batches across GPUs. Increasing instance type to a single larger GPU (e.g., p3.16xlarge) helps but is less effective than multi-GPU distribution. Hyperparameter tuning doesn't directly reduce training time. Spot instances may interrupt.

Answer

Increase the learning rate

Answer

Switch to a larger instance type, such as ml.p3.16xlarge

Answer

Use managed Spot Training

Question 5

A data scientist builds a Random Forest model using SageMaker. The model performs well on training data but poorly on test data. Which step is most likely to reduce overfitting?

Accepted Answer

Reduce the maximum depth of each tree. Reducing the maximum depth of each tree limits the complexity of individual decision trees, preventing them from memorizing noise and specific patterns in the training data. This directly addresses overfitting by enforcing simpler, more generalized splits, which improves performance on unseen test data.

Answer

Increase the number of trees

Answer

Switch to a linear model

Answer

Increase the number of features considered at each split

Question 6

A healthcare company needs to predict patient readmission risk using clinical notes. Which AWS service can be used to preprocess the text data into numerical features for a machine learning model?

Accepted Answer

Amazon Comprehend. Amazon Comprehend is a natural language processing (NLP) service that can extract entities, key phrases, and sentiment. It is suitable for preprocessing clinical notes into features. SageMaker Ground Truth is for data labeling. Rekognition is for images. Translate is for translation.

Answer

Amazon SageMaker Ground Truth

Answer

Amazon Translate

Answer

Amazon Rekognition

Question 7

A data scientist uses SageMaker Autopilot to automatically build a binary classification model. The dataset has 50 features and 100,000 rows. After the experiment, Autopilot provides multiple candidate models. Which candidate should the data scientist select to minimize inference latency for real-time predictions?

Accepted Answer

The model that is a linear learner. SageMaker Autopilot explores various algorithms including linear models, tree-based ensembles, and neural networks. For real-time inference with low latency, simpler models like linear or logistic regression or shallow decision trees are preferred. XGBoost with many trees or deep neural networks increase latency.

Answer

The model with the smallest memory footprint

Answer

The model with the highest validation accuracy

Answer

The model with the lowest validation loss

Question 8

A company uses SageMaker to train a time-series forecasting model using Amazon Forecast. The dataset contains historical sales data for 10,000 products over 2 years. Which data format is required for the target time series?

Accepted Answer

A CSV file with columns: timestamp, target_value, item_id. Amazon Forecast requires the target time series data to be in a CSV format with specific columns: timestamp, target_value, and item_id. This structured format allows the service to correctly identify the time series for each product and the target metric to forecast. The CSV format is the standard input for Forecast's built-in algorithms and ensures compatibility with the dataset import process.

Answer

A single JSON file with nested arrays

Answer

A text file with one value per line

Answer

A Parquet file partitioned by date

Question 9

A data scientist needs to implement a recommendation system for an e-commerce website. Which Amazon service is specifically designed for building and deploying recommendation models?

Accepted Answer

Amazon Personalize. Amazon Personalize is a fully managed machine learning service that provides real-time personalized recommendations. It is purpose-built for recommendation systems. SageMaker is a general-purpose ML platform, but Personalize is specialized for recommendations.

Answer

Amazon SageMaker

Answer

Amazon Rekognition

Answer

Amazon Forecast

Question 10

A data scientist trains a neural network using TensorFlow on SageMaker. The training job fails with a 'CUDA out of memory' error. What is the most likely cause and solution?

Accepted Answer

The model is too large for the GPU. Use a smaller batch size.. CUDA out of memory indicates that the GPU memory is insufficient for the batch size or model size. Reducing the batch size is a common fix. Switching to CPU is not ideal for deep learning. Increasing the number of instances may help but requires distributed training setup. Upgrading to a larger instance type is another option, but reducing batch size is simpler.

Answer

The dataset is too large. Use SageMaker Pipe mode.

Answer

The training script has a bug. Use SageMaker Debugger.

Answer

The instance type is insufficient. Use distributed training across multiple instances.

Question 11

Which TWO metrics are suitable for evaluating a regression model? (Select TWO.)

Accepted Answer

Root Mean Squared Error (RMSE). Root Mean Squared Error (RMSE) is a standard metric for regression models because it measures the average magnitude of prediction errors in the same units as the target variable. It penalizes larger errors more heavily due to squaring, making it sensitive to outliers and useful for comparing model performance.

Answer

Accuracy

Answer

F1-score

Answer

Precision

Question 12

Which THREE techniques help reduce overfitting in a neural network? (Select THREE.)

Accepted Answer

Dropout. Dropout randomly drops units during training, L2 regularization penalizes large weights, and early stopping halts training when validation error increases. Data augmentation can also help but is not listed. Batch normalization may help but primarily for training stability.

Answer

Increasing the number of layers

Answer

Using a larger batch size

Question 13

Which TWO services can be used to serve machine learning models for real-time inference? (Select TWO.)

Accepted Answer

Amazon SageMaker. Amazon SageMaker provides real-time endpoints. AWS Lambda can be used with a custom container for inference. Amazon Rekognition is a specialized service for image analysis, not general model serving. Amazon Comprehend is NLP service. Amazon Athena is query service.

Answer

Amazon Rekognition

Answer

Amazon Athena

Answer

Amazon Comprehend

Question 14

Refer to the exhibit. An IAM policy is attached to a SageMaker execution role. A data scientist tries to create a training job that reads training data from s3://my-bucket/confidential/data.csv. What will happen?

Accepted Answer

The training job will fail with an access denied error. The policy allows s3:GetObject on my-bucket/* but explicitly denies s3:GetObject on my-bucket/confidential/*. Since explicit Deny overrides any Allow, the training job will fail with an access denied error.

Answer

The training job will succeed because there is an Allow on my-bucket/*

Answer

The training job will succeed because the Deny statement is invalid

Answer

The training job will fail because the role lacks sagemaker:CreateTrainingJob

Question 15

Refer to the exhibit. A SageMaker training job failed with the error shown. What is the most likely cause of this error?

Accepted Answer

The input data contains missing or invalid values. The error indicates that the input data contains NaN or infinite values. This is a data quality issue. The algorithm expects clean numeric values. The algorithm itself is fine; the training script may have a bug but the error specifically points to input data.

Answer

The training algorithm is not compatible with the data type

Answer

The training instance type is not powerful enough

Answer

The training script has a syntax error