MLS-C01 Modeling Practice Test 21 — 15 Questions

Question 1

A company is training a deep learning model on Amazon SageMaker using a large dataset stored in S3. The training job is failing with an error indicating insufficient memory. The model architecture and hyperparameters are fixed. Which change is MOST likely to resolve the issue without modifying the model code?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Enable SageMaker's distributed data parallelism.. Enable SageMaker's distributed data parallelism. Since the model architecture and hyperparameters are fixed, the insufficient memory error likely arises because the dataset is too large to fit into the memory of a single instance. Distributed data parallelism splits the training data across multiple instances, allowing each instance to process a smaller subset, thereby reducing per-instance memory usage without any code modifications. Option B (managed Spot training) reduces cost but does not address memory. Option C (using a larger instance) could provide more memory but may be more expensive and does not directly solve the root cause of data size relative to fixed hyperparameters. Option D (Pipe mode) improves data streaming efficiency but does not reduce the memory required for model parameters or intermediate activations.

Answer

Use managed Spot training to get cheaper compute.

Answer

Use a larger instance type with more memory.

Answer

Use Pipe mode for input data instead of File mode.

Question 2

A data scientist is using Amazon SageMaker to train a gradient boosting model on a dataset with categorical features. The dataset contains a column 'UserID' with over 1 million unique values. The training is taking very long and the model size is large. Which technique would MOST effectively reduce training time and model size while maintaining accuracy?

Accepted Answer

Apply feature hashing to UserID.. Hashing reduces the number of distinct categories to a fixed number of buckets, controlling dimensionality. Option A is wrong because one-hot encoding would explode the feature space. Option C is wrong because label encoding creates ordinal relationships that may mislead the model. Option D is wrong because removing UserID likely loses important signal.

Answer

Use one-hot encoding on UserID.

Answer

Use label encoding for UserID.

Answer

Remove UserID from the dataset.

Question 3

A machine learning engineer is using Amazon SageMaker to deploy a model for real-time inference. The model must respond within 100 milliseconds. The initial deployment uses a single ml.m5.large instance, but latency is too high. Which change should the engineer make to reduce latency?

Accepted Answer

Switch to a compute-optimized instance like ml.c5.2xlarge.. A compute-optimized instance like ml.c5.2xlarge provides more CPU and memory, reducing inference latency. Option B is wrong because batch transform is for offline predictions, not real-time; it does not reduce latency for real-time inference. Option C is wrong because using a smaller instance (ml.t2.medium) reduces resources and would likely increase latency, not reduce it. Option D is wrong because multi-model endpoints share resources among models and can lead to contention, potentially increasing latency.

Answer

Use batch transform instead of real-time endpoint.

Answer

Deploy to a single ml.t2.medium instance to reduce cost.

Answer

Deploy the model on a multi-model endpoint.

Question 4

A data scientist is training a binary classification model using Amazon SageMaker's built-in XGBoost algorithm. The dataset is highly imbalanced (95% negative class, 5% positive class). The model achieves high accuracy but poor recall on the positive class. Which TWO actions should the data scientist take to improve recall without significantly sacrificing precision?

Accepted Answer

Set scale_pos_weight to the ratio of negative to positive samples.. Options B and E are correct. Using scale_pos_weight adjusts the weight of the positive class, directly addressing imbalance. SMOTE oversamples the minority class to balance the dataset. Option A is wrong because subsampling the majority class may lose information. Option C is wrong because increasing max_depth may overfit. Option D is wrong because reducing eta may slow convergence but not directly help imbalance.

Answer

Perform random undersampling of the majority class.

Answer

Increase the max_depth hyperparameter.

Answer

Reduce the learning rate (eta) and increase num_round.

Question 5

A company uses Amazon SageMaker to train a linear regression model. During evaluation, they observe that the model has high bias (underfitting). Which THREE actions can reduce bias?

Accepted Answer

Add polynomial features.. Options B, C, and E are correct. Bias (underfitting) occurs when the model is too simple to capture patterns in the data. Adding polynomial features (B) increases model complexity, allowing the linear regression to fit non-linear relationships. Reducing regularization strength (C) reduces the penalty on large coefficients, letting the model fit the training data more closely. Using a random forest model (E) is a more complex algorithm capable of capturing non-linear patterns, thus reducing bias. Option A (increasing L2 regularization) increases bias by penalizing large weights. Option D (using a smaller training dataset) typically increases bias due to less data.

Answer

Increase L2 regularization.

Answer

Use a smaller training dataset.

Question 6

A machine learning engineer is deploying a model using Amazon SageMaker. The model requires preprocessing steps (e.g., scaling, encoding) that were applied during training. Which TWO options can ensure the same preprocessing is applied at inference?

Accepted Answer

Create a Scikit-learn pipeline that includes preprocessing and the model, then deploy it.. Options D and E are correct. A Scikit-learn pipeline bundles preprocessing and the model into a single object, ensuring consistent preprocessing during training and inference. SageMaker Inference Pipeline chains a preprocessing container with the model container, allowing separate preprocessing steps to be applied consistently at inference time. Option A is wrong because using a Lambda function can introduce inconsistencies if not carefully managed, and it adds latency. Option B is wrong because a separate preprocessing endpoint adds complexity and may not guarantee identical preprocessing logic. Option C is wrong because retraining the model per inference request is impractical and computationally expensive.

Answer

Implement preprocessing as an AWS Lambda function invoked before inference.

Answer

Deploy a separate preprocessing endpoint and call it before the model endpoint.

Answer

Retrain the model in each inference request with the preprocessing applied.

Question 7

Refer to the exhibit. A data scientist is trying to run a SageMaker training job using a script that reads training data from 's3://my-bucket/training/data.csv'. The job fails with an access denied error. What is the MOST likely reason?

Accepted Answer

The S3 bucket policy may deny access, or the IAM role lacks necessary permissions beyond GetObject.. The most likely reason for the access denied error is that the S3 bucket policy may deny access, or the IAM role used by SageMaker lacks the necessary permissions beyond s3:GetObject. The script reads from a specific path, but the role might not have permissions to list the bucket or access other required objects. Option B is incorrect because the error is about access denied, not VPC endpoint configuration. Option C is incorrect because the CreateTrainingJob action is allowed. Option D is incorrect because the S3 path is correctly formatted.

Answer

The training job is running in a VPC without S3 VPC endpoint.

Answer

The sagemaker:CreateTrainingJob action is not allowed on the specific resource.

Answer

The S3 path is incorrectly formatted.

Question 8

A company uses Amazon SageMaker to train a custom TensorFlow model for image classification. The training job runs on a single ml.p3.2xlarge instance. The dataset contains 500,000 images stored in S3. The training time is too long (over 24 hours). The data scientist wants to reduce training time without changing the model architecture. The dataset is already in TFRecord format. The training script uses the default TensorFlow data pipeline. Which change will MOST significantly reduce training time?

Accepted Answer

Use SageMaker's distributed data parallelism with multiple instances.. Using SageMaker's distributed data parallelism with multiple instances increases the number of GPUs and splits the training data across them, directly reducing the compute time. Option A is incorrect because simply increasing the number of data files does not reduce the computational workload, and Pipe mode primarily helps with streaming data but does not accelerate model training. Option C is incorrect because switching from File to Pipe mode improves data loading but does not address the core compute bottleneck. Option D is incorrect because optimizing the data pipeline with tf.data.Dataset.prefetch and cache can improve I/O efficiency, but the most significant performance gain comes from scaling out the training across multiple GPUs.

Answer

Use SageMaker Pipe mode and increase the number of data files.

Answer

Switch the input mode from File to Pipe.

Answer

Optimize the data pipeline using tf.data.Dataset.prefetch and cache.

Question 9

A data scientist is using Amazon SageMaker Autopilot to automatically build a binary classification model. The dataset has 50 features and 100,000 rows. After the experiment completes, the best candidate model achieves an F1 score of 0.85 on the validation set. However, when deployed to a real-time endpoint, the model's F1 score drops to 0.72 on production data. The data distributions between training and production are similar. What is the MOST likely cause of the performance drop?

Accepted Answer

The Autopilot pipeline used features that are not available at inference time (data leakage).. Data leakage during Autopilot's feature engineering can lead to overly optimistic validation scores because features may have been derived from the target variable or from future information. Option A is wrong because similar distributions suggest no drift. Option B is wrong because Autopilot handles missing values. Option C is irrelevant because inference instance type does not affect model accuracy.

Answer

Concept drift occurred between training and production.

Answer

The production data contains missing values that were not present in training.

Answer

The inference endpoint uses a different instance type than training.

Question 10

A company uses Amazon SageMaker to train a time-series forecasting model using the built-in DeepAR algorithm. The training data consists of daily sales for 1000 products over 2 years. The model performs well on most products, but for a few products with intermittent demand (sporadic sales), the predictions are poor. Which action should the data scientist take to improve predictions for these products?

Accepted Answer

Create a separate forecasting model specifically for intermittent demand products, using a model designed for such patterns (e.g., Croston's method).. Intermittent demand patterns (sporadic sales) require specialized models like Croston's method, which are designed to handle non-continuous demand. Option B is wrong because a simple linear regression model cannot capture the irregular spikes of intermittent demand; such models assume continuous, steady patterns. Option C is wrong because increasing the context length of DeepAR, which is built for regular time series, does not address the fundamental issue of sporadic demand—the model still expects continuous values. Option D is wrong because adding unrelated product categories introduces noise and does not help the model learn the specific intermittent pattern of the target products.

Answer

Use a linear regression model for all products.

Answer

Increase the context length of the DeepAR model to capture longer history.

Answer

Add more training data by including additional product categories.

Question 11

A data scientist is using Amazon SageMaker to train a linear learner model for regression. After reviewing the training logs, the data scientist notices that the loss is not decreasing and remains high. The learning rate is set to 0.01. The data is normalized. What should the data scientist do to improve convergence?

Accepted Answer

Try different learning rates, such as 0.001 or 0.1.. The loss is not decreasing, which often indicates an inappropriate learning rate. Trying different learning rates (e.g., 0.001 or 0.1) can help find a rate that allows the model to converge. Option A is incorrect because the data is already normalized; normalizing again would have no effect. Option B is incorrect because reducing the mini-batch size introduces more stochasticity but does not directly address learning rate issues. Option D is incorrect because increasing epochs does not help if the learning rate prevents convergence; the model may continue to oscillate or plateau.

Answer

Normalize the data again.

Answer

Reduce the mini-batch size.

Answer

Increase the number of epochs.

Question 12

A company uses Amazon SageMaker to host a model for real-time inference. The model is a large ensemble that takes 2 seconds to load into memory. To reduce cold start latency, the data scientist uses SageMaker's managed warm pools. However, they notice that during a sudden traffic spike, new instances still experience high latency. What is the BEST way to ensure consistently low latency for all requests?

Accepted Answer

Configure auto scaling based on the number of active invocations to maintain a buffer of warmed instances.. Configuring auto scaling based on the number of active invocations maintains a buffer of warmed instances. This ensures that when traffic spikes occur, new instances are already loaded and ready to serve requests, avoiding cold start latency. Option A is wrong because using a larger instance type does not eliminate cold starts; the model still needs to load into memory. Option C is wrong because reducing instances increases the frequency of cold starts. Option D is wrong because SageMaker Serverless Inference has its own cold start overhead and is not suitable for workloads requiring consistently low latency.

Answer

Use a larger instance type to reduce model loading time.

Answer

Reduce the number of instances to minimize cold start frequency.

Answer

Switch to SageMaker Serverless Inference.

Question 13

A data scientist is training a text classification model using Amazon SageMaker's BlazingText algorithm. The dataset consists of 1 million documents, each labeled with one of 10 categories. The model achieves 92% accuracy on a held-out test set. However, when deployed, the model performs poorly on documents containing slang and typos. What should the data scientist do to improve model robustness?

Accepted Answer

Augment the training data by introducing common slang replacements and typos.. Data augmentation by introducing common slang replacements and typos into the training data increases the model's robustness to such variations, helping it generalize better to real-world text that contains slang and typos. Removing such documents (Option A) reduces the training data and does not teach the model to handle these variations. Increasing the embedding dimension (Option C) or number of epochs (Option D) does not directly address the issue of slang and typos.

Answer

Remove all documents with slang or typos from the training set.

Answer

Increase the embedding dimension from 100 to 300.

Answer

Increase the number of training epochs.

Question 14

A data scientist is using Amazon SageMaker to train a model using the built-in XGBoost algorithm. The training job is taking a long time. The data scientist notices that the input data is in CSV format and the training job is using File mode. The data size is 50 GB. What is the BEST way to reduce training time?

Accepted Answer

Switch the input mode to Pipe.. Pipe mode streams data directly from S3 to the training algorithm, reducing I/O overhead and improving throughput compared to File mode, which downloads the entire dataset first. Option A (larger instance) may not help if the bottleneck is I/O rather than compute. Option B (Parquet format) can improve performance but is not as impactful as Pipe mode for streaming. Option C (reduce features) could reduce training time but at the cost of model accuracy, making it not the best approach.

Answer

Use a larger instance type with more vCPUs.

Answer

Convert the data to Parquet format.

Answer

Reduce the number of features in the dataset.

Question 15

A data scientist is training a binary classification model to predict customer churn. The dataset has 10,000 samples with 500 churners (5% positive class). Which TWO techniques should the scientist use to address the class imbalance? (Choose TWO.)

Accepted Answer

Use SMOTE to oversample the minority class. (SMOTE) generates synthetic samples for the minority class, effectively balancing the dataset. Option E (class_weight='balanced') adjusts the loss function to penalize misclassifications of the minority class more heavily. Option B (tuning threshold after training) is a post-processing step, not a technique to address imbalance during training. Option C (random undersampling) can discard useful data, leading to loss of information. Option D (oversampling by duplication) can cause overfitting due to repeated copies of the same samples.

Answer

Tune the decision threshold after training

Answer

Randomly undersample the majority class to match minority size

Answer

Oversample the minority class by duplicating existing samples