MLA-C01 ML Model Development Practice Test 7 — 15 Questions

Question 1

A team is using Amazon SageMaker's Automatic Model Tuning (AMT) to optimize hyperparameters for a random forest model. After 10 training jobs, the best objective metric value plateaus. The team wants to explore the search space more broadly. Which AMT strategy should they use?

Accepted Answer

Random search. Random search samples hyperparameters randomly and covers the search space more broadly, which can help escape a plateau. Bayesian optimization focuses on promising regions, which may not explore broadly. Grid search is exhaustive but expensive. Hyperband uses early stopping to allocate resources efficiently but still may not explore broadly if the plateau persists.

Answer

Grid search

Answer

Bayesian optimization

Answer

Hyperband

Question 2

A data scientist is splitting a dataset into training and test sets. Which two practices should they follow? (Select TWO.)

Accepted Answer

Shuffle the data before splitting. Shuffling the data before splitting ensures randomness and prevents ordering biases. Ensuring the test set is representative of real-world distribution (e.g., using stratified sampling) improves generalization evaluation. An 80/20 split is common but not always optimal; 50/50 is not recommended. Stratified sampling is a specific technique for classification, but the general practice of representativeness is broader.

Answer

Use stratified sampling for classification to preserve class proportions

Answer

Use a 50/50 split to maximize test data

Answer

Use a 80/20 split for large datasets

Question 3

A machine learning engineer is training a neural network using Amazon SageMaker. The training job uses a single GPU instance. To improve training speed using distributed training, which two steps should they take? (Select TWO.)

Accepted Answer

Modify the training script to use Horovod or PyTorch DistributedDataParallel. Distributed training requires both modifying the training script to use a distributed framework (e.g., Horovod, PyTorch DDP) and increasing the number of instances. Splitting the dataset into smaller files can improve I/O but is not about distribution. SageMaker's distributed data parallelism library is one option, but modifying the script with a framework is the general step. Automatic mixed precision improves speed on a single GPU but does not enable distributed training.

Answer

Split the dataset into smaller files

Answer

Use SageMaker's distributed data parallelism library

Answer

Enable automatic mixed precision

Question 4

A data scientist is developing a gradient boosting model and observes that the model is overfitting to the training data. Which three techniques can help reduce overfitting? (Select THREE.)

Accepted Answer

Reduce the learning rate. Reducing the learning rate slows down learning and helps reduce overfitting. Subsampling (data/feature sampling) adds randomness and reduces overfitting. Early stopping stops training before overfitting occurs. Increasing the number of trees or tree depth increases model complexity, worsening overfitting. Increasing regularization parameters (like lambda, alpha) also helps reduce overfitting, but the three most common for gradient boosting are reducing learning rate, subsampling, and early stopping.

Answer

Increase the maximum depth of trees

Answer

Increase the regularization parameters (e.g., lambda, alpha)

Answer

Increase the number of trees

Question 5

Refer to the exhibit. A data scientist ran a training job using a custom algorithm container. The job failed with the error shown. What is the most likely cause?

Accepted Answer

The algorithm script references an undefined variable or metric named 'loss'. The error 'Cannot evaluate expression: loss' indicates that the training script attempted to compute or log a variable named 'loss' that is not defined in the code. The training image access, S3 output path, and instance type are not related to this specific error.

Answer

The S3 output path is incorrect

Answer

The training image is not accessible

Answer

The instance type is insufficient

Question 6

Refer to the exhibit. A data scientist configured an automatic model tuning job for a classification model. The tuning job completed after 20 training jobs, but the best validation accuracy was only 0.65. What is the most effective way to potentially improve the result?

Accepted Answer

Increase MaxNumberOfTrainingJobs to 100. With only 20 training jobs, Bayesian optimization may not have fully explored the hyperparameter space. Increasing the maximum number of training jobs allows more exploration and increases the chance of finding better hyperparameters. Changing to random search could help but Bayesian is generally more efficient. Changing the objective to training accuracy would not improve generalization. Increasing parallel jobs does not increase total exploration.

Answer

Change the strategy to Random

Answer

Change the objective metric to training:accuracy

Answer

Increase MaxParallelTrainingJobs to 10

Question 7

Refer to the exhibit. A data scientist configured SageMaker Debugger to monitor training for overfitting. However, the rule never triggers even though the model appears to be overfitting. What is the most likely reason?

Accepted Answer

The debug hook is not collecting the validation loss. The DebugHookConfig only collects losses and gradients. The overfitting rule likely compares training loss to validation loss, but validation loss is not being collected. Without a collection for validation loss (e.g., validation:loss), the rule cannot evaluate the condition for overfitting. The rule evaluator image, instance type, and S3 path are less likely causes; the image is a placeholder but might be valid.

Answer

The instance type for the rule is too small

Answer

The S3 output path is not writable

Answer

The rule evaluator image is incorrect

Question 8

A company is training a deep learning model on Amazon SageMaker. The training job started but has been stuck in 'InProgress' state for an unusually long time with low CPU utilization. The data scientist suspects a bottleneck. What should be the first troubleshooting step?

Accepted Answer

Review CloudWatch Logs for the training container to identify errors or warnings.. Option A is correct because checking CloudWatch logs for the training container can reveal errors like resource limits or data loading issues. Option B is wrong because restarting without investigation may waste time. Option C is wrong because using Spot instances does not address the stuck job. Option D is wrong because increasing instance count may not help if the bottleneck is elsewhere.

Answer

Switch the training job to use Spot instances to reduce cost and potentially improve throughput.

Answer

Increase the number of training instances to parallelize data loading.

Answer

Stop and restart the training job with a different instance type.

Question 9

A data engineer needs to split a time-series dataset into training and validation sets for a forecasting model. Which split method should be used to avoid data leakage?

Accepted Answer

Temporal split where training uses data up to a cutoff date and validation uses later data.. Option B is correct because for time-series data, a temporal split ensures that validation data comes from a later time period than training data. Option A is wrong because random splits can cause future data to leak into training. Option C is wrong while cross-validation is useful, it must be done in a time-aware manner (e.g., rolling origin), but standard k-fold cross-validation is not appropriate. Option D is wrong because feature importance is not a splitting method.

Answer

Use k-fold cross-validation with random shuffling.

Answer

Use feature importance scores to weight the splitting process.

Answer

Random split with 80% training and 20% validation.

Question 10

A machine learning engineer runs a SageMaker HyperparameterTuningJob with Bayesian optimization strategy. The job terminates earlier than the specified MaxNumberOfTrainingJobs. The engineer notices that the best objective metric value has not improved for several consecutive jobs. What is the most likely adjustment to make?

Accepted Answer

Adjust the early stopping tolerance (e.g., increase the number of consecutive jobs with no improvement allowed).. Option C is correct because Bayesian optimization uses early stopping to avoid wasting resources on unpromising hyperparameters. The early stopping tolerance can be configured to be less aggressive. Option A is wrong because increasing max jobs would still not help if the search gets stuck. Option B is wrong because decreasing the number of hyperparameters may reduce the search space but does not address early stopping. Option D is wrong because grid search is less efficient and would ignore the ongoing Bayesian optimization.

Answer

Switch to a grid search strategy to cover all hyperparameter combinations.

Answer

Increase the MaxNumberOfTrainingJobs parameter to allow more exploration.

Answer

Decrease the number of hyperparameters being tuned.

Question 11

A company deploys a real-time inference endpoint on SageMaker for a customer-facing application. Traffic patterns are unpredictable and sometimes spike. The endpoint must scale automatically to handle load while minimizing cost. Which approach should the company take?

Accepted Answer

Configure a target tracking scaling policy on the endpoint using CloudWatch metrics.. Option D is correct because a target tracking scaling policy with a metric like average latency or request count scales automatically based on demand. Option A is wrong because using multiple models does not address scaling. Option B is wrong because increasing instance type without auto-scaling leads to over-provisioning. Option C is wrong because batch transform is for asynchronous, not real-time.

Answer

Switch to batch transform for all inference requests.

Answer

Use a larger instance type to handle peak traffic.

Answer

Deploy multiple models behind an Application Load Balancer.

Question 12

A data scientist trains a neural network on SageMaker using the TensorFlow framework. The training accuracy is lower than expected, and the scientist suspects vanishing gradients. How can the scientist leverage SageMaker Debugger to diagnose this?

Accepted Answer

Use a built-in Debugger rule to monitor gradient magnitudes during training.. Option A is correct because SageMaker Debugger includes built-in rules such as vanishing_gradient and exploding_gradient that automatically monitor tensors. Option B is wrong because TensorBoard is not integrated with SageMaker Debugger directly for rule-based alerts. Option C is wrong because adding more epochs may not solve vanishing gradients. Option D is wrong because reducing learning rate can help but does not diagnose the issue.

Answer

Increase the number of training epochs to allow gradients to propagate.

Answer

Export model summaries to TensorBoard for manual inspection.

Answer

Reduce the learning rate to prevent gradient explosion.

Question 13

A machine learning team wants to detect bias in a binary classification model before deployment. They use SageMaker Clarify. Which type of bias metric should they compute to understand whether the model treats different demographic groups unfairly in predictions?

Accepted Answer

Pre-training bias metrics like Class Imbalance (CI) and Difference in Positive Proportions in Labels (DPPL).. Option B is correct because pre-training bias metrics (such as class imbalance, Kullback-Leibler divergence) reveal data bias before modeling, while post-training metrics assess prediction bias. Option A is wrong because feature importance explains model behavior but not bias. Option C is wrong because SHAP values are for model interpretability, not bias metrics per se. Option D is wrong because retraining does not detect bias.

Answer

SHAP (SHapley Additive exPlanations) values from the test predictions.

Answer

A re-run of the training job with a fairness constraint.

Answer

Feature importance values after training.

Question 14

A data scientist wants to train an XGBoost model using the SageMaker Python SDK with a custom training script. Which estimator class should be used?

Accepted Answer

sagemaker.xgboost.estimator.XGBoost with a script mode entry point.. Option C is correct because the SageMaker XGBoost framework estimator allows users to bring their own training script while using the optimized XGBoost container. Option A is wrong because the built-in XGBoost algorithm does not support custom scripts; it expects a specific input format. Option B is wrong because scikit-learn estimator does not natively support XGBoost training. Option D is wrong because TensorFlow estimator is for TensorFlow models.

Answer

sagemaker.sklearn.SKLearnEstimator.

Answer

sagemaker.tensorflow.TensorFlowEstimator.

Answer

sagemaker.xgboost.XGBoostEstimator with the built-in algorithm mode.

Question 15

A team uses SageMaker Experiments to track multiple training runs. They need to register the best-performing model in the model registry for approval. Which method ensures the model artifacts and metadata are captured correctly?

Accepted Answer

Use the SageMaker Model Registry's create_model_package_from_estimator or equivalent API to register the model.. Option D is correct because SageMaker Model Registry provides a centralized catalog for model versions with associated metadata, metrics, and approval status. Option A is wrong because manual comparison is error-prone. Option B is wrong because Experiments track runs but do not natively register models. Option C is wrong because Lambda is not a direct mechanism for model registration.

Answer

Write an AWS Lambda function to copy the best model to a specific S3 prefix.

Answer

Manually download the best model artifact and upload to S3, then create a model in SageMaker.

Answer

Use Experiment analytics to view results and then create a model package using the Run's artifact URI.