MLS-C01 Modeling Practice Test 32 — 15 Questions

Question 1

A data scientist is using Amazon SageMaker Autopilot to automatically build a model for a regression problem. The dataset has 100 features and 50,000 rows. Autopilot recommends a model with an R² of 0.85 on the validation set. However, when deployed to production, the model performs poorly (R² of 0.2). What is the most likely cause?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The production data distribution has shifted from the training data distribution. A large gap between validation performance (R² 0.85) and production performance (R² 0.2) with no change to the model itself is the classic signature of data drift — the production input distribution no longer matches the training/validation distribution. Autopilot's validation set is a random holdout from the same historical dataset, so it cannot detect drift that occurs after deployment. The fix is to monitor for drift (e.g., SageMaker Model Monitor) and retrain on recent data.

Answer

The model is overfitting to the training data

Answer

The model is underfitting due to insufficient training

Answer

Autopilot selected the wrong features

Question 2

A data scientist is training a recurrent neural network (RNN) for time series forecasting. The training loss decreases steadily for the first 10 epochs, then plateaus. The validation loss starts increasing after epoch 10. What is the most appropriate action?

Accepted Answer

Stop training early and use the model from epoch 10. The described pattern — training loss still decreasing while validation loss begins rising after epoch 10 — is textbook overfitting. The model is memorizing training data rather than generalizing. Early stopping at epoch 10 (or restoring the checkpoint with the lowest validation loss) is the standard, low-cost remedy and prevents further degradation of validation performance.

Answer

Continue training for more epochs

Answer

Add more layers to the network

Answer

Increase the batch size

Question 3

A data scientist is using Amazon SageMaker to train a linear regression model. The dataset has outliers. Which TWO techniques can help reduce the impact of outliers? (Choose TWO.)

Accepted Answer

Trim the dataset to remove extreme values. Option A is correct because trimming (or winsorizing) the dataset removes or caps extreme values, so the linear regression fit is no longer dominated by outliers that would otherwise pull the coefficients disproportionately. Option D is correct because Huber loss behaves quadratically for small residuals but linearly for large residuals, so outliers contribute bounded gradients and have far less influence on the learned coefficients than with squared error. Option B is not appropriate because adding features does not address the influence of outliers and can even worsen overfitting. Option C is not appropriate because L1 regularization penalizes coefficient magnitude to induce sparsity, not to down-weight outlier residuals. Option E is not appropriate because standardization only rescales features to zero mean and unit variance; it does not reduce the leverage or residual impact of outliers.

Answer

Add more features

Answer

Apply L1 regularization

Answer

Standardize the features

Question 4

A data scientist is tuning a random forest model using SageMaker Hyperparameter Tuning. The objective metric is validation:accuracy. Which THREE hyperparameters are most commonly tuned for random forest? (Choose THREE.)

Accepted Answer

Minimum samples per leaf (min_samples_leaf). The three hyperparameters most commonly tuned for a random forest are min_samples_leaf (B), max_depth (C), and n_estimators (D). Minimum samples per leaf (B) controls the minimum number of samples required to be at a leaf node, directly affecting the bias-variance tradeoff and preventing overfitting on small leaves. Maximum depth (C) limits how deep each decision tree can grow, which governs model complexity and helps regularize the ensemble. Number of trees (n_estimators) (D) determines how many trees are built in the forest, and increasing it generally improves accuracy up to a point of diminishing returns. Learning rate (A) is a gradient-boosting hyperparameter (e.g., XGBoost), not a random forest parameter, and batch size (E) is a training/optimization setting for neural networks or mini-batch gradient descent, not applicable to random forest tuning.

Answer

Learning rate

Answer

Batch size

Question 5

A company is using SageMaker to train a TensorFlow model for image classification. The training is slow on a single GPU instance. Which TWO strategies can reduce training time? (Choose TWO.)

Accepted Answer

Use SageMaker Pipe Mode for data ingestion. Option B is correct because SageMaker Pipe Mode streams training data directly from Amazon S3 into the training container instead of downloading the full dataset to local disk first, which removes the I/O bottleneck and keeps the GPU fed with data faster, reducing overall training time. Option D is correct because distributed training across multiple GPUs (e.g., using Horovod or SageMaker's distributed data parallel libraries) splits the compute and gradient synchronization work across devices, so each epoch completes in a fraction of the time compared to a single GPU. Option A is incorrect because increasing image size raises the computational cost per sample, making training slower, not faster. Option C is incorrect because adding more epochs increases total training time rather than reducing it. Option E is incorrect because decreasing the batch size means more forward/backward passes per epoch, which typically slows training and underutilizes GPU parallelism.

Answer

Increase the image size

Answer

Increase the number of training epochs

Answer

Decrease the batch size

Question 6

A data science team is using Amazon SageMaker to train a deep learning model for object detection using the built-in SSD algorithm. The dataset consists of 100,000 labeled images stored in a SageMaker Pipe Mode input. The training job uses a single ml.p3.2xlarge instance. After 2 hours, the training job fails with the error 'ResourceLimitExceeded: The account-level service limit for ml.p3.2xlarge for training job usage is 1. Contact AWS Support to request a limit increase'. However, the team has already submitted a limit increase request and it was approved for 5 instances. What is the most likely cause of the error?

Accepted Answer

The service limit increase has not yet been applied to the account in the current region. The error 'ResourceLimitExceeded' indicates that the account's service limit for ml.p3.2xlarge training instances has been exceeded. Even though the team requested and received approval for a limit increase to 5 instances, the increase may not have taken effect yet in the current region. AWS service limit increases are applied per region, and there can be a propagation delay after approval. Option A (GPU memory) would cause a different error such as 'OutOfMemory'. Option B (algorithm requirement) is unrelated because the SSD algorithm does run on the chosen instance. Option D (S3 permissions) would result in an 'AccessDenied' error, not a limit error. Therefore, option C is the correct answer.

Answer

The instance is running out of GPU memory

Answer

The built-in SSD algorithm requires a GPU instance type with at least 16 GB of GPU memory

Answer

The IAM role does not have permission to access the S3 bucket for model artifacts

Question 7

A startup is deploying a machine learning model for real-time recommendation on Amazon SageMaker. The model is a TensorFlow model (1 GB) and the endpoint uses a single ml.c5.2xlarge instance. The inference latency is currently 500 ms per request. The startup expects traffic to increase 10x in the next month. They want to maintain latency under 500 ms. What is the most cost-effective solution?

Accepted Answer

Set up auto-scaling for the endpoint based on average latency or request count. Setting up auto-scaling for the SageMaker endpoint based on average latency or request count allows the endpoint to automatically add instances as traffic increases, maintaining latency under 500 ms. This is cost-effective because it scales out only when needed and scales in during low traffic, avoiding over-provisioning.

Answer

Use SageMaker Batch Transform to process requests in batches

Answer

Switch to a GPU instance type for faster inference

Answer

Upgrade to a larger CPU instance type, such as ml.c5.4xlarge

Question 8

A company uses Amazon SageMaker to train a linear regression model. During evaluation, they observe that the model has high bias (underfitting). Which THREE actions can reduce bias?

Accepted Answer

Add polynomial features.. Option B (Add polynomial features) is correct because high bias means the linear hypothesis class is too simple to capture the underlying relationship; adding polynomial features expands the feature space so the model can fit nonlinear patterns and reduce underfitting. Option C (Reduce the regularization strength) is correct because L2 regularization penalizes large weights and shrinks the model toward a simpler solution; lowering the regularization coefficient (for example, reducing alpha in the SageMaker LinearLearner predictor) allows the model more flexibility to fit the training data and lowers bias. Option E (Use a random forest model instead of linear regression) is correct because a random forest is a higher-capacity, non-linear ensemble that can represent complex decision boundaries a linear model cannot, thereby reducing bias. Option A is incorrect because increasing L2 regularization further constrains the model and increases bias, not reduces it. Option D is incorrect because using a smaller training dataset reduces the information available to the model, typically worsening underfitting rather than alleviating it.

Answer

Increase L2 regularization.

Answer

Use a smaller training dataset.

Question 9

Refer to the exhibit. A data scientist is trying to run a SageMaker training job using a script that reads training data from 's3://my-bucket/training/data.csv'. The job fails with an access denied error. What is the MOST likely reason?

Accepted Answer

The S3 bucket policy may deny access, or the IAM role lacks necessary permissions beyond GetObject.. The most likely reason for an access denied error when a SageMaker training job reads from S3 is that the SageMaker execution IAM role lacks the necessary S3 permissions, or an explicit bucket policy denies access. SageMaker uses the execution role to access S3, so missing s3:GetObject or a restrictive bucket policy will cause the failure.

Answer

The training job is running in a VPC without S3 VPC endpoint.

Answer

The sagemaker:CreateTrainingJob action is not allowed on the specific resource.

Answer

The S3 path is incorrectly formatted.

Question 10

A company uses Amazon SageMaker to train a custom TensorFlow model for image classification. The training job runs on a single ml.p3.2xlarge instance. The dataset contains 500,000 images stored in S3. The training time is too long (over 24 hours). The data scientist wants to reduce training time without changing the model architecture. The dataset is already in TFRecord format. The training script uses the default TensorFlow data pipeline. Which change will MOST significantly reduce training time?

Accepted Answer

Use SageMaker's distributed data parallelism with multiple instances.. Using SageMaker's distributed data parallelism with multiple instances increases the number of GPUs and splits the training data across them, directly reducing the compute time. Option A is incorrect because simply increasing the number of data files does not reduce the computational workload, and Pipe mode primarily helps with streaming data but does not accelerate model training. Option C is incorrect because switching from File to Pipe mode improves data loading but does not address the core compute bottleneck. Option D is incorrect because optimizing the data pipeline with tf.data.Dataset.prefetch and cache can improve I/O efficiency, but the most significant performance gain comes from scaling out the training across multiple GPUs.

Answer

Use SageMaker Pipe mode and increase the number of data files.

Answer

Switch the input mode from File to Pipe.

Answer

Optimize the data pipeline using tf.data.Dataset.prefetch and cache.

Question 11

A company uses Amazon SageMaker to train a time-series forecasting model using the built-in DeepAR algorithm. The training data consists of daily sales for 1000 products over 2 years. The model performs well on most products, but for a few products with intermittent demand (sporadic sales), the predictions are poor. Which action should the data scientist take to improve predictions for these products?

Accepted Answer

Create a separate forecasting model specifically for intermittent demand products, using a model designed for such patterns (e.g., Croston's method).. DeepAR is a deep learning model that works well for time series with continuous or count data, but it assumes a certain regularity and may struggle with intermittent demand where many zero values occur. Croston's method is specifically designed for intermittent demand by separately forecasting the demand size and the inter-demand interval, then combining them. Thus, creating a separate model for these products using a method like Croston's is the most appropriate action.

Answer

Use a linear regression model for all products.

Answer

Increase the context length of the DeepAR model to capture longer history.

Answer

Add more training data by including additional product categories.

Question 12

A data scientist is using Amazon SageMaker Autopilot to automatically build a model for a regression problem. The dataset has 100 features and 50,000 rows. Autopilot recommends a model with an R² of 0.85 on the validation set. However, when deployed to production, the model performs poorly (R² of 0.2). What is the most likely cause?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Try different learning rates, such as 0.001 or 0.1.. When loss is not decreasing and remains high despite normalized data, the most likely culprit is an inappropriate learning rate. A learning rate of 0.01 may be too high (causing divergence/oscillation) or too low (causing slow convergence). Trying values like 0.001 or 0.1 is the standard diagnostic step to find a rate that allows the optimizer to descend the loss surface effectively.

Answer

Normalize the data again.

Answer

Reduce the mini-batch size.

Answer

Increase the number of epochs.

Question 13

A company uses Amazon SageMaker to host a model for real-time inference. The model is a large ensemble that takes 2 seconds to load into memory. To reduce cold start latency, the data scientist uses SageMaker's managed warm pools. However, they notice that during a sudden traffic spike, new instances still experience high latency. What is the BEST way to ensure consistently low latency for all requests?

Accepted Answer

Configure auto scaling based on the number of active invocations to maintain a buffer of warmed instances.. SageMaker managed warm pools keep instances initialized with the model loaded, but they only help when instances are already in the pool. During a traffic spike, auto scaling must proactively maintain a buffer of warmed instances based on invocation metrics so new capacity is ready before requests arrive. Configuring target tracking on InvocationsPerInstance with a sufficient warm pool size ensures pre-warmed instances absorb the spike without cold starts.

Answer

Use a larger instance type to reduce model loading time.

Answer

Reduce the number of instances to minimize cold start frequency.

Answer

Switch to SageMaker Serverless Inference.

Question 14

A data scientist is training a text classification model using Amazon SageMaker's BlazingText algorithm. The dataset consists of 1 million documents, each labeled with one of 10 categories. The model achieves 92% accuracy on a held-out test set. However, when deployed, the model performs poorly on documents containing slang and typos. What should the data scientist do to improve model robustness?

Accepted Answer

Augment the training data by introducing common slang replacements and typos.. Augmenting the training data with slang replacements and typos directly addresses the distribution shift between the training set and real-world documents. By exposing the model to these variations during training, it learns to generalize to noisy text, improving robustness. This is a data-centric approach that targets the root cause: the model has not seen such patterns before.

Answer

Remove all documents with slang or typos from the training set.

Answer

Increase the embedding dimension from 100 to 300.

Answer

Increase the number of training epochs.

Question 15

A data scientist is using Amazon SageMaker Autopilot to automatically build a model for a regression problem. The dataset has 100 features and 50,000 rows. Autopilot recommends a model with an R² of 0.85 on the validation set. However, when deployed to production, the model performs poorly (R² of 0.2). What is the most likely cause?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Switch the input mode to Pipe.. Switching from File mode to Pipe mode streams training data directly from S3 into the algorithm without downloading the full dataset to the training instance's EBS volume first. For a 50 GB CSV dataset, File mode incurs significant download time before training begins, whereas Pipe mode starts training almost immediately and reduces both startup latency and disk space requirements. This is the standard SageMaker optimization for large datasets.

Answer

Use a larger instance type with more vCPUs.

Answer

Convert the data to Parquet format.

Answer

Reduce the number of features in the dataset.