MLS-C01 Modeling Practice Test 41 — 15 Questions

Question 1

A company is building a real-time fraud detection system using Amazon SageMaker. The model is a gradient boosting classifier trained on 500 GB of transactional data. The inference endpoint is deployed as a SageMaker real-time endpoint using an ml.c5.9xlarge instance. The model is serialized using the native format of the framework (XGBoost). The endpoint receives about 100 requests per second with an average payload size of 10 KB. The company observes that the endpoint's latency is around 200 ms, but they need under 100 ms. The data scientist profiles the endpoint and finds that the model inference time is 50 ms, but the remaining time is spent on data preprocessing and serialization/deserialization. The preprocessing involves converting JSON input to a NumPy array and then to a DMatrix. Which action is most likely to reduce latency to meet the requirement?

Accepted Answer

Use a more efficient serialization format such as Apache Arrow or Protocol Buffers for the input data. Option D is correct. By using SageMaker Batch Transform, the company can process requests in batches, reducing per-request overhead. However, the requirement is for real-time, so this may not be suitable. Option A is wrong because larger instances may not reduce preprocessing overhead. Option B is wrong because reducing model complexity could hurt accuracy. Option C is wrong, but it's a plausible approach: using a more efficient serialization format (e.g., Protocol Buffers) can reduce deserialization time. Actually, option C is correct: using a more efficient data format reduces preprocessing time. Option D is wrong because batch transform is for asynchronous, not real-time. The correct answer should be C. Let me re-evaluate: The stem says 'remaining time is spent on data preprocessing and serialization/deserialization.' Using a more efficient serialization format (e.g., Protobuf instead of JSON) can reduce overhead. Option A: upgrading instance may not help if the bottleneck is serialization. Option B: reducing model complexity may affect accuracy. Option D: batch transform is not real-time. So C is best.

Answer

Switch to SageMaker Batch Transform to process requests in batches

Answer

Use a larger instance type such as ml.c5.18xlarge

Answer

Reduce the number of trees in the model

Question 2

A data scientist is using Amazon SageMaker to train a natural language processing model using a custom Docker container. The training script reads data from an S3 bucket and writes checkpoints to an S3 bucket. The training job is failing with the error 'Unable to write to checkpoint path: s3://my-bucket/checkpoints/'. The IAM role associated with the training job has the following policy: {'Effect': 'Allow', 'Action': 's3:PutObject', 'Resource': 'arn:aws:s3:::my-bucket/checkpoints/*'}. The bucket 'my-bucket' exists and the prefix 'checkpoints/' is empty. What is the most likely cause of the failure?

Accepted Answer

The IAM role is missing the s3:ListBucket permission. The error 'Unable to write to checkpoint path' occurs because the SageMaker training job's IAM role lacks the `s3:ListBucket` permission. Even though the role has `s3:PutObject` on the checkpoint prefix, SageMaker's S3 client first performs a `ListObjects` (or `HeadObject`) call to verify the bucket exists and to check the prefix state before writing. Without `s3:ListBucket` on the bucket itself, the API call fails, causing the write operation to abort.

Answer

The IAM role does not have s3:PutObject permission

Answer

The S3 bucket does not exist

Answer

The checkpoint prefix already contains objects

Question 3

A company is using Amazon SageMaker to deploy a machine learning model that predicts equipment failure. The model is a binary classifier that outputs a probability. The company wants to set a threshold such that the model correctly identifies 95% of actual failures (recall >= 0.95). The model's precision at the current threshold of 0.5 is 0.7. The data scientist evaluates the model on a test set and obtains the following confusion matrix at threshold 0.5: TP=95, FN=5, FP=40, TN=860. The total actual positives are 100. Which threshold adjustment should the data scientist make to achieve the recall goal?

Accepted Answer

Keep the threshold at 0.5. Option B is correct. Lowering the threshold increases recall by classifying more instances as positive, which reduces false negatives. Currently recall is 95/100 = 0.95, so recall is already 95%. Actually, recall is already 95% at threshold 0.5. So the requirement is already met. But the question might imply that recall needs to be at least 95%, which it is. However, the stem says 'the company wants to set a threshold such that the model correctly identifies 95% of actual failures (recall >= 0.95)'. At threshold 0.5, recall is 95/100 = 0.95, so it meets the requirement. So no adjustment is needed. But the options include 'Keep the threshold at 0.5' as option D. So D is correct. Let me check: If recall is already 0.95, then no change needed. So answer D.

Answer

Decrease the threshold to 0.1

Answer

Increase the threshold to 0.7

Answer

Decrease the threshold to 0.3

Question 4

A data scientist is using Amazon SageMaker to train a deep learning model using a built-in algorithm. The training job uses an ml.p3.2xlarge instance and takes 10 hours to complete. The scientist wants to reduce training time without changing the algorithm or model architecture. The instance's GPU utilization is consistently at 95%, but CPU utilization is only 20%. The data input pipeline uses SageMaker Pipe mode with the 'TrainingInputMode' set to 'Pipe'. The training dataset is 200 GB in CSV format stored in S3. Which approach is most likely to reduce training time?

Accepted Answer

Use a larger instance type with more GPUs, such as ml.p3.8xlarge. Option D is correct. Since GPU utilization is high (95%), the GPU is the bottleneck. Upgrading to a more powerful GPU instance (e.g., p3.8xlarge with 4 GPUs) can reduce training time by parallelizing computation. Option A is wrong because File mode may not help and could increase I/O overhead. Option B is wrong because Pipe mode is already being used. Option C is wrong because reducing batch size could underutilize GPU further.

Answer

Switch from Pipe mode to File mode to reduce I/O overhead

Answer

Use Pipe mode with 'S3DataType' as 'AugmentedManifestFile'

Answer

Reduce the batch size to improve GPU utilization

Question 5

A data scientist is using Amazon SageMaker to train a linear regression model. The training data contains 100 features and 1 million rows. The scientist notices that the model is overfitting, with training R² of 0.99 and validation R² of 0.65. The scientist has already tried adding L2 regularization and reducing the number of features. Which additional technique should the scientist try to reduce overfitting?

Accepted Answer

Increase the amount of training data. Option C is correct. Adding more training data can help reduce overfitting by providing a more representative sample. Option A is wrong because increasing model complexity (more features) would worsen overfitting. Option B is wrong because increasing learning rate may cause instability. Option D is wrong because increasing batch size may not help and could lead to overfitting.

Answer

Increase the batch size

Answer

Increase the learning rate

Answer

Add more features

Question 6

A company uses Amazon SageMaker to host a model for real-time inference. The model is a large ensemble of 10 deep learning models, each 500 MB. The total model size is 5 GB, which exceeds the 5 GB limit for SageMaker real-time endpoints. The data scientist wants to reduce the model size without significantly impacting accuracy. The ensemble uses averaging of predictions from all models. The scientist has access to a validation set with 10,000 samples. Which technique should the scientist use to reduce the model size?

Accepted Answer

Use model distillation to train a smaller model that approximates the ensemble. Option A is correct. Model distillation trains a smaller student model to mimic the ensemble, reducing size while preserving accuracy. Option B is wrong because price-aware instance selection does not reduce model size. Option C is wrong because SageMaker Neo is for optimization, not size reduction below 5 GB. Option D is wrong because pruning alone may not reduce size enough.

Answer

Use a more expensive instance type to host the model

Answer

Use SageMaker Neo to compile and optimize the model

Answer

Apply weight pruning to each model in the ensemble

Question 7

A data scientist is building a binary classification model to predict customer churn. The dataset has 10,000 samples with 500 churners (positive class). Which TWO techniques should be used to address the class imbalance? (Choose 2.)

Accepted Answer

Use random undersampling of the majority class. SMOTE and undersampling are standard techniques for handling class imbalance.

Answer

Use a higher learning rate during training

Answer

Use L1 regularization on the model

Answer

Use principal component analysis (PCA) to reduce dimensionality

Question 8

A machine learning engineer is training a deep learning model on Amazon SageMaker. The training job is taking a long time. Which THREE actions can reduce training time? (Choose 3.)

Accepted Answer

Use SageMaker managed spot training. Warm pools, distributed training, and spot training can reduce training time.

Answer

Use a smaller batch size

Answer

Use SageMaker hyperparameter tuning jobs

Question 9

A data scientist is training a random forest model for regression. The model shows high variance on the validation set. Which TWO actions are most likely to reduce variance? (Choose 2.)

Accepted Answer

Decrease the maximum depth of trees. Decreasing depth and increasing min samples per leaf both reduce complexity, lowering variance.

Answer

Use bootstrap sampling with replacement

Answer

Increase the number of trees in the forest

Answer

Increase the number of features considered at each split

Question 10

A startup is building a recommendation system for an e-commerce platform using collaborative filtering. They have a dataset of user-item interactions (ratings) with 1 million users and 100,000 items. The data is sparse (99% missing ratings). They need to train a model on Amazon SageMaker that can handle large-scale sparse data efficiently. Which approach should they use?

Accepted Answer

Use the built-in Factorization Machines algorithm in SageMaker. SageMaker's Factorization Machines handle sparse data efficiently and are designed for recommendation tasks.

Answer

Use PCA to reduce dimensionality and then apply k-nearest neighbors

Answer

Use the built-in XGBoost algorithm with one-hot encoding for user and item IDs

Answer

Implement a neural network with dense layers using the built-in MXNet framework

Question 11

A financial services company is developing a fraud detection model using gradient boosting. The dataset contains 10 million transactions with 0.1% fraudulent. The model is trained on a SageMaker ml.m5.2xlarge instance and takes 8 hours. The team needs to reduce training time without sacrificing model performance. They have permission to use up to 4 instances. What should they do?

Accepted Answer

Switch to a built-in XGBoost with GPU support and use a p3.2xlarge instance. GPU instances like p3.2xlarge accelerate XGBoost training substantially.

Answer

Use SageMaker hyperparameter tuning to find faster hyperparameters

Answer

Use SageMaker's distributed training with data parallelism across 4 ml.m5.2xlarge instances

Answer

Use SageMaker managed spot training with checkpointing

Question 12

A healthcare company is building a model to predict patient readmission within 30 days. They have structured electronic health records (EHR) data with 200 features. The data includes missing values, categorical variables with high cardinality (e.g., diagnosis codes), and a severe class imbalance (5% readmission). They need to deploy a model on SageMaker that is interpretable and achieves high recall for the positive class. Which combination of techniques should they use?

Accepted Answer

Use XGBoost with SMOTE, feature selection via SHAP, and deploy as a SageMaker endpoint. XGBoost with SMOTE and SHAP balances interpretability and performance.

Answer

Use logistic regression with one-hot encoding and random undersampling

Answer

Use PCA for dimensionality reduction, then train a linear SVM with class weights

Answer

Use a deep neural network with embeddings for categorical variables and oversample the minority class

Question 13

A data scientist is training a linear regression model on a dataset with 50 features. After training, they notice that the model performs well on training data but poorly on test data. They suspect overfitting. Which action should they take to reduce overfitting?

Accepted Answer

Add L2 regularization (Ridge regression). Regularization is a standard technique to combat overfitting in linear models.

Answer

Use a larger learning rate

Answer

Add more features to the model

Answer

Increase the number of training epochs

Question 14

A company is using SageMaker built-in object detection algorithm to detect defects in manufacturing images. The model is trained on 10,000 labeled images and achieves 95% accuracy. However, in production, the model misclassifies many defective items as non-defective (false negatives). The business requires recall > 90% for the defect class. Which action should they take?

Accepted Answer

Adjust the decision threshold of the model to increase recall at the expense of precision. Threshold tuning directly optimizes recall for a given class.

Answer

Use a different algorithm such as semantic segmentation

Answer

Use SageMaker's Automatic Model Tuning to find better hyperparameters

Answer

Retrain the model with more images of non-defective items

Question 15

A media company uses SageMaker to train a neural network for content recommendation. The model uses embeddings for users and items. Training is slow and they want to reduce time. The dataset has 10 million users and 1 million items. They have a cluster of 8 p3.16xlarge instances. Which strategy is most likely to reduce training time?

Accepted Answer

Use SageMaker's model parallelism to split the embedding layers across GPUs. Model parallelism is designed for large models with memory-intensive layers like embeddings.

Answer

Use data parallelism to replicate the model on each GPU and synchronize gradients

Answer

Reduce the embedding dimension from 256 to 64

Answer

Use a smaller batch size to fit on each GPU