MLS-C01 Modeling Practice Test 23 — 15 Questions

Question 1

A machine learning engineer is training a regression model to predict house prices using Amazon SageMaker. The dataset contains 10,000 samples and 50 numerical features. After training a linear regression model, the engineer notices that the training loss is low, but the validation loss is high. The engineer suspects overfitting. The dataset is already normalized. Which action should the engineer take to reduce overfitting?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Add L2 regularization (weight decay) to the loss function.. L2 regularization (weight decay) adds a penalty term to the loss function that discourages large weight values, effectively reducing model complexity and overfitting. Option A (increasing learning rate) can cause the model to diverge or overshoot minima, and does not directly prevent overfitting. Option B (PCA) reduces the number of features, which can help with overfitting but may discard important information; regularization is a more targeted approach for linear models. Option D (decreasing mini-batch size) introduces more noise into gradient estimates, which can sometimes act as a regularizer but is less effective and reliable than L2 regularization for this scenario.

Answer

Increase the learning rate to speed up convergence.

Answer

Reduce the number of features using PCA.

Answer

Decrease the mini-batch size during training.

Question 2

A data scientist is training a binary classification model on a highly imbalanced dataset where the positive class represents only 1% of the data. The model achieves 99% accuracy but only identifies 5% of the actual positives. Which metric should the data scientist use to evaluate model performance?

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual positives correctly identified by the model. With only 5% of positives detected, recall is 0.05, which directly reveals the model's failure to capture the minority class despite high accuracy. In imbalanced datasets, accuracy is misleading because the model can achieve 99% accuracy by simply predicting the majority class (negative) for all instances.

Answer

Mean squared error

Answer

Accuracy

Answer

Precision

Question 3

A team is deploying a real-time inference endpoint using Amazon SageMaker. The model is a large deep learning model that requires GPU for inference. The endpoint must handle variable traffic patterns with minimal latency. Which deployment strategy should the team use?

Accepted Answer

Use a SageMaker multi-model endpoint with GPU instance type.. B is correct because SageMaker multi-model endpoints (MMEs) allow multiple models to be hosted on a single GPU-backed endpoint, dynamically loading and unloading models from disk to GPU memory as needed. This reduces cost and cold-start latency compared to single-model endpoints, while still providing GPU acceleration for deep learning inference. MMEs are ideal for variable traffic patterns because they can scale horizontally and share GPU resources efficiently.

Answer

Deploy a single model endpoint with an auto-scaling policy.

Answer

Deploy a serverless endpoint using SageMaker Serverless Inference.

Answer

Use SageMaker Batch Transform to process requests in batches.

Question 4

A machine learning engineer is training a regression model to predict house prices using Amazon SageMaker. The dataset contains 10,000 samples and 50 numerical features. After training a linear regression model, the engineer notices that the training loss is low, but the validation loss is high. The engineer suspects overfitting. The dataset is already normalized. Which action should the engineer take to reduce overfitting?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use Pipe input mode instead of File input mode.. Pipe input mode streams data directly from S3 to the training instances without first downloading it to the local EBS volume, eliminating the I/O bottleneck of reading a 500 GB CSV file. This reduces the time spent on data loading and allows the XGBoost algorithm to begin training sooner, which is especially beneficial for large datasets.

Answer

Convert the data to Parquet format.

Answer

Increase the number of instances to 20.

Answer

Increase the size of the EBS volume attached to each instance.

Question 5

A data scientist is building a regression model to predict house prices. The dataset has 10 features, and the model shows high variance with a low bias. Which technique should the data scientist use to reduce variance?

Accepted Answer

Apply L2 regularization to the model.. L2 regularization (Ridge regression) penalizes large coefficients by adding a squared magnitude term to the loss function, which shrinks the model's weights and reduces variance without substantially increasing bias. This directly addresses the high-variance, low-bias symptom, making the model less sensitive to fluctuations in the training data.

Answer

Increase the depth of decision trees in the ensemble.

Answer

Add more features to the model.

Answer

Reduce the amount of training data.

Question 6

A company is deploying a fraud detection model using Amazon SageMaker. The model is a linear learner trained on 100 GB of data. For inference, the model receives individual transactions and must return a prediction within 100 ms. Which endpoint configuration should the team use to meet the latency requirement?

Accepted Answer

Deploy a single model endpoint using a GPU instance and enable autoscaling.. A single-model endpoint on a GPU instance provides the low-latency, high-throughput inference required for real-time fraud detection. GPU instances accelerate linear learner inference by parallelizing matrix operations, enabling sub-100 ms predictions for individual transactions. Autoscaling ensures the endpoint can handle traffic spikes without degrading latency.

Answer

Use a multi-model endpoint with CPU instances.

Answer

Use a batch transform job scheduled every minute.

Answer

Deploy using SageMaker Serverless Inference.

Question 7

A data scientist is using SageMaker to train a TensorFlow model. The training script uses tf.data.Dataset to load data from S3. Training is slow because of I/O bottleneck. Which change should the data scientist make to improve I/O performance?

Accepted Answer

Use Pipe input mode for the training channel.. Pipe input mode streams data directly from S3 into the training algorithm without writing to disk, eliminating the I/O bottleneck caused by downloading entire files. This is particularly effective with tf.data.Dataset, as the pipeline can consume data incrementally, reducing latency and improving throughput for large datasets.

Answer

Enable EBS optimization on the training instance.

Answer

Use SageMaker local mode for training.

Answer

Convert the dataset to RecordIO format.

Question 8

A data scientist is using Amazon SageMaker to train a model, but the training job fails with an 'Out of memory' error. The instance type is ml.p3.2xlarge. Which action should the data scientist take to resolve the issue?

Accepted Answer

Reduce the mini-batch size in the training script.. The 'Out of memory' error on a single ml.p3.2xlarge instance indicates that the GPU memory is insufficient for the current workload. Reducing the mini-batch size directly decreases the memory footprint per training step, allowing the model to fit within the available GPU memory without changing the instance type or incurring additional costs.

Answer

Use Pipe input mode.

Answer

Increase the number of instances.

Answer

Use a Spot instance.

Question 9

A company uses Amazon SageMaker to deploy a model for real-time inference. The endpoint uses an ml.m5.large instance with automatic scaling based on CPU utilization. The team notices that during traffic spikes, the endpoint returns 5xx errors. What should the team do to improve the endpoint's availability?

Accepted Answer

Increase the instance type to ml.c5.2xlarge.. Upgrading the instance type from ml.m5.large to ml.c5.2xlarge provides more CPU and memory resources, which directly addresses the root cause of 5xx errors during traffic spikes — insufficient compute capacity to handle the request load. Automatic scaling based on CPU utilization may not react quickly enough to sudden spikes, leading to request queuing and timeouts that manifest as 5xx errors. A larger instance type increases the baseline throughput, reducing the likelihood of resource exhaustion before scaling can take effect.

Answer

Reduce the scaling cooldown period.

Answer

Place an Application Load Balancer in front of the endpoint.

Answer

Use Amazon API Gateway to throttle requests.

Question 10

A data scientist is training a model using SageMaker's built-in XGBoost algorithm with a large dataset stored in CSV format. The training job is using File mode. The data scientist wants to reduce the time it takes to start training. Which approach would be most effective?

Accepted Answer

Use Pipe mode for the input data channel.. Pipe mode streams data directly from Amazon S3 into the training container, eliminating the need to first download the entire dataset to the EBS volume. This reduces the startup time significantly because training can begin as soon as the first records arrive, rather than waiting for the full download to complete.

Answer

Increase the size of the EBS volume.

Answer

Convert the data to Parquet format.

Answer

Increase the number of training instances.

Question 11

A data scientist is training a binary classifier using imbalanced data. Which TWO techniques can help improve model performance on the minority class? (Choose two.)

Accepted Answer

Use the F1 score as the evaluation metric.. The F1 score is the harmonic mean of precision and recall, making it a robust evaluation metric for imbalanced datasets because it captures both false positives and false negatives. Unlike accuracy, which can be misleadingly high when the majority class dominates, the F1 score provides a balanced measure of model performance on the minority class.

Answer

Undersample the majority class randomly.

Answer

Use accuracy as the evaluation metric.

Answer

Apply L1 regularization to the model.

Question 12

A company is using SageMaker to deploy a model for real-time inference. The model requires GPU for low latency. Which THREE configurations should the company consider for high availability and cost optimization? (Choose three.)

Accepted Answer

Use a multi-model endpoint to share GPU instances among multiple models.. A multi-model endpoint allows multiple models to be hosted on the same GPU-backed instance, sharing the GPU resources and reducing idle time. This improves cost efficiency by maximizing GPU utilization while still providing low-latency inference for each model. It is a recommended pattern for serving many models with GPU requirements without provisioning separate endpoints.

Answer

Use Spot instances for the endpoint.

Answer

Use SageMaker Batch Transform for inference.

Question 13

A data scientist is using SageMaker to train a deep learning model. The training job runs on a single GPU instance and is taking too long. Which THREE actions can the data scientist take to reduce training time? (Choose three.)

Accepted Answer

Switch to Pipe input mode to reduce I/O waiting time.. Pipe input mode streams training data directly from Amazon S3 to the GPU instance, reducing I/O waiting time compared to the default File mode, which downloads the entire dataset to the EBS volume first. This minimizes the time the GPU spends idle waiting for data, especially for large datasets that cannot fit entirely in memory.

Answer

Increase the size of the EBS volume attached to the instance.

Answer

Increase the number of instances but keep the same total data.

Question 14

A company is deploying a machine learning model for real-time fraud detection. The model must respond within 100ms. Which SageMaker endpoint deployment strategy should be used?

Accepted Answer

Deploy the model to a SageMaker Real-Time Inference endpoint with a Multi-Model Endpoint configuration.. A SageMaker Real-Time Inference endpoint with a Multi-Model Endpoint configuration provides low-latency (sub-100ms) responses by keeping models loaded in memory and routing requests efficiently. This architecture is ideal for real-time fraud detection where multiple models may be needed, and it meets the strict latency requirement without the cold-start overhead of serverless options.

Answer

Deploy the model to a SageMaker Serverless Inference endpoint.

Answer

Deploy the model as an AWS Lambda function with an API Gateway trigger.

Answer

Use SageMaker Batch Transform to process requests in batches.

Question 15

A data scientist is training a binary classification model on imbalanced data (95% negative, 5% positive). Which metric is most appropriate for evaluating model performance?

Accepted Answer

Area Under the ROC Curve (AUC-ROC). AUC-ROC is the most appropriate metric for imbalanced binary classification because it evaluates the model's ability to distinguish between positive and negative classes across all classification thresholds, without being biased by the 95% negative majority. It measures the trade-off between true positive rate and false positive rate, making it robust to class imbalance.

Answer

R-squared

Answer

Mean Squared Error (MSE)

Answer

Accuracy