MLS-C01 Modeling Practice Test 15 — 15 Questions

Question 1

A company runs a real-time recommendation system on SageMaker with a model that uses a deep neural network. The endpoint uses a single ml.p3.2xlarge instance. Recently, the number of users has grown, and the endpoint's latency has increased from 50ms to 200ms, exceeding the SLA of 100ms. The model inference code is optimized and cannot be improved further. The company wants to reduce latency while minimizing cost. The data scientist has the following options: A. Switch to a larger instance type with more GPU memory, such as ml.p3.8xlarge. B. Use SageMaker's Elastic Inference to attach an EI accelerator to the existing instance. C. Deploy the model on multiple smaller instances (e.g., ml.p3.2xlarge) behind a load balancer and distribute traffic. D. Convert the model to use TensorFlow Lite and deploy on a CPU-based instance. Which option is the MOST cost-effective and meets the latency requirement?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use SageMaker's Elastic Inference. The most cost-effective option is B (Use SageMaker's Elastic Inference) because it provides dedicated GPU acceleration at a fraction of the cost of a full GPU instance, reducing inference latency without requiring a larger instance. Option C (Switch to a larger instance type) would increase cost significantly. Option D (Deploy on multiple smaller instances behind a load balancer) would increase complexity and cost, and may not guarantee latency reduction. Option A (Convert to TensorFlow Lite on CPU) could reduce cost but may not meet the latency requirement as CPU inference is slower than GPU for deep neural networks, and model conversion might impact accuracy.

Answer

Convert to TensorFlow Lite on CPU

Answer

Switch to a larger instance type, e.g., ml.p3.8xlarge

Answer

Deploy on multiple smaller instances behind a load balancer

Question 2

A data scientist is training a binary classification model on a dataset with 100 features and 10,000 rows. The model overfits significantly: training accuracy is 99%, but validation accuracy is 80%. The data scientist has tried L1 and L2 regularization without improvement. The dataset is clean and representative. Which approach is MOST likely to reduce overfitting? A. Increase the number of training epochs. B. Add more training data by generating synthetic samples using SMOTE. C. Reduce the number of features using PCA. D. Use a simpler model like logistic regression instead of a decision tree ensemble. The data scientist needs to maintain a validation accuracy above 85%, but the current model is too complex. The company has limited budget for data labeling. Which option is BEST?

Accepted Answer

Use a simpler model like logistic regression. The current model (likely a decision tree ensemble like Random Forest or XGBoost) is too complex for the dataset, causing overfitting. Switching to a simpler model like logistic regression reduces variance by limiting the hypothesis space, which directly addresses overfitting without requiring additional data or feature engineering. Given the limited labeling budget, this approach is cost-effective and can improve generalization, potentially achieving the required >85% validation accuracy.

Answer

Add more training data by generating synthetic samples using SMOTE

Answer

Reduce the number of features using PCA

Answer

Increase the number of training epochs

Question 3

A company is using SageMaker to train a text classification model using a built-in BlazingText algorithm. The dataset has 500,000 documents, each labeled with one of 10 categories. The training time is taking longer than expected. The data scientist wants to speed up training without increasing cost. The training job is using a single ml.m4.xlarge instance. The code uses default hyperparameters. Which change is MOST likely to reduce training time? A. Use a larger instance type, such as ml.m4.4xlarge. B. Increase the learning rate significantly. C. Use SageMaker Managed Spot Training. D. Use the 'mode' hyperparameter set to 'batch_skipgram' instead of 'supervised'. The company has a fixed budget and wants to minimize cost while reducing training time. Which option should the data scientist choose?

Accepted Answer

Use SageMaker Managed Spot Training. The best because SageMaker Managed Spot Training uses Spot Instances which are significantly cheaper than On-Demand instances. This cost saving allows the data scientist to launch multiple training instances in parallel within the same fixed budget, thereby reducing training time. While other options either increase cost, risk poor convergence, or change the task to unsupervised, Spot Training provides a cost-effective way to accelerate training without altering the algorithm or instance type.

Answer

Increase the learning rate significantly

Answer

Use the 'mode' hyperparameter set to 'batch_skipgram' instead of 'supervised'

Answer

Use a larger instance type, such as ml.m4.4xlarge

Question 4

A data scientist is training a binary classification model on an imbalanced dataset (95% negative class, 5% positive class). The model achieves 95% accuracy but only predicts the negative class for all examples. Which metric should the scientist use to evaluate model performance more appropriately?

Accepted Answer

AUC-ROC. AUC-ROC is robust to class imbalance because it evaluates the model's ability to discriminate between positive and negative classes across all classification thresholds, rather than relying on a single threshold. In this scenario, the model predicts only the negative class, so its true positive rate is 0 and false positive rate is 0, yielding an AUC-ROC of 0.5 (random performance), which correctly reflects the model's lack of predictive power.

Answer

F1 score

Answer

Mean squared error

Answer

Accuracy

Question 5

A data scientist has this IAM policy attached to an IAM role used by SageMaker. When trying to create a training job, the scientist gets an access denied error. The training data is in 's3://my-bucket/training-data/'. What is the most likely cause?

Accepted Answer

Missing s3:ListBucket permission. The error occurs because the IAM policy grants s3:GetObject permission on the training data objects but lacks s3:ListBucket permission on the bucket itself. SageMaker's CreateTrainingJob API first performs a ListBucket call to verify the bucket exists and to enumerate objects, even if the exact object key is known. Without s3:ListBucket, the ListBucket call fails, resulting in an access denied error.

Answer

The bucket name is misspelled

Answer

The S3 resource ARN is incorrect

Answer

The sagemaker:CreateTrainingJob action is not allowed

Question 6

A data scientist ran a SageMaker training job that failed with the error shown. The training script expects the data in '/opt/ml/input/data/training/train.csv'. What is the most likely issue?

Accepted Answer

The S3 data path should point to the exact file, not the folder. The SageMaker training job expects the S3 data path to point directly to the CSV file (e.g., s3://bucket/train.csv), not to a folder containing the file. When the path points to a folder, SageMaker downloads the folder contents but the training script's hardcoded path '/opt/ml/input/data/training/train.csv' fails because the file is not placed at that exact location—SageMaker copies the file into the channel directory with its original name, but the folder path causes the file to be nested or missing, leading to a file-not-found error.

Answer

The hyperparameter 'sagemaker_program' is misspelled

Answer

The training script has a bug in reading the file

Answer

The channel name should be 'train' instead of 'training'

Question 7

A data scientist is setting up a SageMaker training job and has attached this IAM policy to the execution role. The training job fails with an access denied error when trying to write to the output path 's3://my-bucket/output/model.tar.gz'. What additional permission is needed?

Accepted Answer

s3:ListBucket. The training job fails because SageMaker needs to verify that the output S3 bucket exists before writing to it. The s3:ListBucket permission is required to list the contents of the bucket (or confirm its existence) as part of the write operation. Without this permission, the service cannot validate the bucket, resulting in an access denied error even if s3:PutObject is allowed.

Answer

s3:GetObject for the output path

Answer

s3:DeleteObject

Answer

iam:PassRole on the role itself

Question 8

A company is building a real-time fraud detection system using Amazon SageMaker. The model is a gradient boosting classifier trained on 500 GB of transactional data. The inference endpoint is deployed as a SageMaker real-time endpoint using an ml.c5.9xlarge instance. The model is serialized using the native format of the framework (XGBoost). The endpoint receives about 100 requests per second with an average payload size of 10 KB. The company observes that the endpoint's latency is around 200 ms, but they need under 100 ms. The data scientist profiles the endpoint and finds that the model inference time is 50 ms, but the remaining time is spent on data preprocessing and serialization/deserialization. The preprocessing involves converting JSON input to a NumPy array and then to a DMatrix. Which action is most likely to reduce latency to meet the requirement?

Accepted Answer

Use a more efficient serialization format such as Apache Arrow or Protocol Buffers for the input data. The bottleneck is data preprocessing and serialization/deserialization, not model inference. Using a more efficient serialization format like Apache Arrow or Protocol Buffers reduces the overhead of converting JSON to NumPy arrays and DMatrix, directly cutting the 150 ms spent outside inference. This targets the root cause without changing the model or infrastructure.

Answer

Switch to SageMaker Batch Transform to process requests in batches

Answer

Use a larger instance type such as ml.c5.18xlarge

Answer

Reduce the number of trees in the model

Question 9

A data scientist is using Amazon SageMaker to train a natural language processing model using a custom Docker container. The training script reads data from an S3 bucket and writes checkpoints to an S3 bucket. The training job is failing with the error 'Unable to write to checkpoint path: s3://my-bucket/checkpoints/'. The IAM role associated with the training job has the following policy: {'Effect': 'Allow', 'Action': 's3:PutObject', 'Resource': 'arn:aws:s3:::my-bucket/checkpoints/*'}. The bucket 'my-bucket' exists and the prefix 'checkpoints/' is empty. What is the most likely cause of the failure?

Accepted Answer

The IAM role is missing the s3:ListBucket permission. The error 'Unable to write to checkpoint path' occurs because the SageMaker training job's IAM role lacks the `s3:ListBucket` permission. Even though the role has `s3:PutObject` on the checkpoint prefix, SageMaker's S3 client first performs a `ListObjects` (or `HeadObject`) call to verify the bucket exists and to check the prefix state before writing. Without `s3:ListBucket` on the bucket itself, the API call fails, causing the write operation to abort.

Answer

The IAM role does not have s3:PutObject permission

Answer

The S3 bucket does not exist

Answer

The checkpoint prefix already contains objects

Question 10

A data scientist is using Amazon SageMaker to train a linear regression model. The training data contains 100 features and 1 million rows. The scientist notices that the model is overfitting, with training R² of 0.99 and validation R² of 0.65. The scientist has already tried adding L2 regularization and reducing the number of features. Which additional technique should the scientist try to reduce overfitting?

Accepted Answer

Increase the amount of training data. Increasing the amount of training data provides the model with more examples of the underlying distribution, which helps reduce variance and combat overfitting. With 1 million rows and 100 features, the model may still be memorizing noise; adding more diverse data forces the linear regression to generalize better, improving validation R² without changing the model's capacity.

Answer

Increase the batch size

Answer

Increase the learning rate

Answer

Add more features

Question 11

A company uses Amazon SageMaker to host a model for real-time inference. The model is a large ensemble of 10 deep learning models, each 500 MB. The total model size is 5 GB, which exceeds the 5 GB limit for SageMaker real-time endpoints. The data scientist wants to reduce the model size without significantly impacting accuracy. The ensemble uses averaging of predictions from all models. The scientist has access to a validation set with 10,000 samples. Which technique should the scientist use to reduce the model size?

Accepted Answer

Use model distillation to train a smaller model that approximates the ensemble. Model distillation trains a smaller 'student' model to mimic the outputs of the large ensemble 'teacher' on the validation set, effectively compressing 10 models into one compact model. This directly reduces the 5 GB ensemble below the SageMaker real-time endpoint limit while preserving most of the ensemble's predictive accuracy. It is the standard technique for compressing ensembles without significant accuracy loss.

Answer

Use a more expensive instance type to host the model

Answer

Use SageMaker Neo to compile and optimize the model

Answer

Apply weight pruning to each model in the ensemble

Question 12

A machine learning engineer is training a deep learning model on Amazon SageMaker. The training job is taking a long time. Which THREE actions can reduce training time? (Choose 3.)

Accepted Answer

Use SageMaker managed spot training. Option A (SageMaker managed spot training) is correct because it lets the training job run on spare EC2 capacity, which can reduce cost and, more importantly, allows the job to be resumed from checkpoints, making it practical to use high-throughput instances for long training runs. Option B (SageMaker managed warm pools) is correct because warm pools keep provisioned instances in a ready state between jobs, eliminating the startup/provisioning latency that would otherwise be added to each training job, thereby reducing total training time. Option C (SageMaker distributed training with data parallelism) is correct because it splits the training data across multiple GPUs/instances and synchronizes gradients, directly parallelizing the compute-heavy training loop and shortening wall-clock training time. Option D (smaller batch size) is not correct because reducing the batch size generally increases the number of iterations per epoch and can slow convergence, so it does not reliably reduce training time. Option E (SageMaker hyperparameter tuning jobs) is not correct because tuning launches many separate training jobs to find better hyperparameters; it optimizes model quality rather than reducing the time of a single long-running training job.

Answer

Use a smaller batch size

Answer

Use SageMaker hyperparameter tuning jobs

Question 13

A data scientist is training a random forest model for regression. The model shows high variance on the validation set. Which TWO actions are most likely to reduce variance? (Choose 2.)

Accepted Answer

Decrease the maximum depth of trees. Option B is correct because decreasing the maximum depth of the trees limits how finely each tree can fit the training data, which reduces the model's variance (at the cost of some bias) and helps control overfitting. Option C is correct because increasing the minimum samples per leaf forces each leaf to represent more data points, smoothing the predictions and reducing variance from fitting noise. Option A is not a variance-reduction action in this context; bootstrap sampling with replacement is already a core part of how random forests are built and does not by itself address high variance. Option D is incorrect because adding more trees mainly stabilizes predictions and reduces variance only marginally after a point, and it does not address the underlying overfitting as directly as limiting tree complexity. Option E is incorrect because increasing the number of features considered at each split tends to make trees more correlated and can increase variance rather than reduce it.

Answer

Use bootstrap sampling with replacement

Answer

Increase the number of trees in the forest

Answer

Increase the number of features considered at each split

Question 14

A startup is building a recommendation system for an e-commerce platform using collaborative filtering. They have a dataset of user-item interactions (ratings) with 1 million users and 100,000 items. The data is sparse (99% missing ratings). They need to train a model on Amazon SageMaker that can handle large-scale sparse data efficiently. Which approach should they use?

Accepted Answer

Use the built-in Factorization Machines algorithm in SageMaker. Factorization Machines are designed specifically for sparse, high-dimensional data and are a standard algorithm for collaborative filtering recommendation systems. SageMaker's built-in Factorization Machines algorithm handles large-scale sparse matrices efficiently and can learn latent factor interactions between users and items. This directly addresses the 1 million users by 100,000 items sparse rating matrix.

Answer

Use PCA to reduce dimensionality and then apply k-nearest neighbors

Answer

Use the built-in XGBoost algorithm with one-hot encoding for user and item IDs

Answer

Implement a neural network with dense layers using the built-in MXNet framework

Question 15

A financial services company is developing a fraud detection model using gradient boosting. The dataset contains 10 million transactions with 0.1% fraudulent. The model is trained on a SageMaker ml.m5.2xlarge instance and takes 8 hours. The team needs to reduce training time without sacrificing model performance. They have permission to use up to 4 instances. What should they do?

Accepted Answer

Switch to a built-in XGBoost with GPU support and use a p3.2xlarge instance. GPU instances like p3.2xlarge accelerate XGBoost training substantially.

Answer

Use SageMaker hyperparameter tuning to find faster hyperparameters

Answer

Use SageMaker's distributed training with data parallelism across 4 ml.m5.2xlarge instances

Answer

Use SageMaker managed spot training with checkpointing