MLA-C01 ML Model Development Practice Test 5 — 15 Questions

Question 1

A data scientist is training an XGBoost model on a large dataset using a SageMaker Training Job. They want to minimize costs without sacrificing model performance. Which instance type and training strategy should they choose?

Accepted Answer

Use a single ml.g4dn.xlarge Spot instance with no distributed training. Using Spot instances with Managed Spot Training can reduce costs by up to 90% compared to On-Demand, and SageMaker automatically handles interruptions. For single-instance training, a single ml.g4dn.xlarge provides sufficient compute for moderate-sized datasets.

Answer

Use a single ml.m5.large On-Demand instance with model parallelism

Answer

Use multiple ml.trn1.2xlarge On-Demand instances with data parallelism

Answer

Use a single ml.p3.2xlarge On-Demand instance with data parallelism

Question 2

A team is fine-tuning a large language model (LLM) using SageMaker and wants to reduce memory footprint during training. Which technique should they use?

Accepted Answer

Use QLoRA (Quantized Low-Rank Adaptation) with 4-bit quantization. QLoRA (Quantized Low-Rank Adaptation) combines 4-bit quantization with low-rank adapters, significantly reducing GPU memory usage while maintaining model quality.

Answer

Use LoRA (Low-Rank Adaptation) with fp32 precision

Answer

Use SageMaker Model Parallelism with tensor parallelism

Answer

Full fine-tuning on a p3.16xlarge instance

Question 3

A machine learning engineer wants to automatically track hyperparameters, metrics, and artifacts for multiple training runs. Which SageMaker feature should they use?

Accepted Answer

SageMaker Experiments. SageMaker Experiments is purpose-built for tracking and comparing training runs, capturing parameters, metrics, and artifacts.

Answer

SageMaker Debugger

Answer

SageMaker Model Monitor

Answer

SageMaker Clarify

Question 4

A data scientist uses SageMaker Automatic Model Tuning (AMT) with Bayesian optimization to tune an XGBoost model. The objective metric is validation:auc, but the tuning job converges to a plateau early. Which action is MOST effective to improve exploration?

Accepted Answer

Increase the exploration_weight parameter in the tuning configuration. Increasing the exploration/exploitation weight (exploration_weight) in Bayesian optimization encourages the algorithm to try more diverse hyperparameter combinations, avoiding premature convergence.

Answer

Increase the number of max parallel jobs

Answer

Decrease the number of hyperparameters being tuned

Answer

Switch the tuning strategy from Bayesian to Random Search

Question 5

A team is training a PyTorch model using SageMaker and wants to use their own custom training container with a specific PyTorch version. Which approach should they use?

Accepted Answer

Use SageMaker Bring Your Own Container (BYOC) with a custom Docker image. BYOC (Bring Your Own Container) allows teams to package their own environment, including custom PyTorch versions, into a Docker container and use it with SageMaker.

Answer

Use the SageMaker built-in PyTorch estimator and set the framework_version

Answer

Use SageMaker Script Mode with a PyTorch script

Answer

Use SageMaker Autopilot to automatically select the container

Question 6

A data scientist trains a binary classification model using SageMaker and obtains an AUC of 0.95 on the test set. However, the precision-recall curve shows low precision for high recall thresholds. The business requires a model that performs well on the minority class. Which metric should the team primarily optimize during hyperparameter tuning?

Accepted Answer

F1-score on the validation set. For imbalanced datasets, the F1-score balances precision and recall, making it a better objective than AUC, which can be misleading when class imbalance exists.

Answer

Accuracy

Answer

AUC (Area Under the ROC Curve)

Answer

Log loss

Question 7

Which SageMaker built-in algorithm should be used for forecasting time series data with seasonal patterns?

Accepted Answer

DeepAR. DeepAR is a supervised learning algorithm for time series forecasting that handles seasonality and trends.

Answer

IP Insights

Answer

BlazingText

Answer

Factorization Machines

Question 8

A data scientist suspects that a deep learning model is overfitting. They enable SageMaker Debugger and want to detect overfitting automatically. Which built-in rule should they use?

Accepted Answer

Overfit. The overfit rule in SageMaker Debugger monitors training and validation loss divergence, a key indicator of overfitting.

Answer

ExplodingGradients

Answer

PoorWeightInitialization

Answer

DeadRelu

Question 9

A company uses SageMaker Autopilot to build a binary classification model. The generated leaderboard shows an ensemble model as the best candidate. The team needs a model that can be deployed for real-time inference with latency < 10ms. What should they do?

Accepted Answer

Select the best single model from the leaderboard (non-ensemble candidate) and deploy it. Autopilot ensemble models may have high latency due to combining multiple models. The team should evaluate non-ensemble candidates (single model) from the leaderboard to meet latency requirements.

Answer

Use SageMaker Inference Recommender to profile the ensemble model and optimize it

Answer

Deploy the ensemble model as a SageMaker endpoint; ensemble models are optimized for low latency

Answer

Retrain the ensemble model with fewer base estimators using a custom container

Question 10

A data scientist is training a linear learner model using SageMaker and notices that the loss is not decreasing. They suspect the issue is exploding gradients. Which SageMaker Debugger rule should they enable to monitor this?

Accepted Answer

ExplodingGradients. The ExplodingGradients rule tracks gradient values and alerts if they exceed a threshold, which is the correct detection for exploding gradients.

Answer

LossNotDecreasing

Answer

VanishingGradients

Answer

Overfit

Question 11

Which SageMaker feature provides AutoML capabilities, including automatic data preprocessing, model selection, and hyperparameter tuning?

Accepted Answer

SageMaker Autopilot. SageMaker Autopilot automates the ML pipeline from data to model, including preprocessing, algorithm selection, and tuning.

Answer

SageMaker Data Wrangler

Answer

SageMaker Automatic Model Tuning

Answer

SageMaker Experiments

Question 12

A machine learning engineer is training a TensorFlow model using SageMaker with distributed training. They need to implement data parallelism across multiple GPUs. Which SageMaker feature should they use to distribute the training?

Accepted Answer

SageMaker Distributed Data Parallelism. SageMaker's distributed data parallelism library handles splitting data across GPUs and synchronizing gradients, optimized for TensorFlow and PyTorch.

Answer

SageMaker Automatic Model Tuning

Answer

SageMaker Debugger

Answer

SageMaker Model Parallelism

Question 13

A data scientist is using SageMaker to train a custom PyTorch model for image classification. They want to use SageMaker Debugger to detect training issues. Which TWO built-in rules are most relevant for detecting common training problems? (Select TWO.)

Accepted Answer

Overfit. ExplodingGradients detects gradients becoming too large, and Overfit detects when validation loss diverges from training loss. Both are common issues.

Answer

DataDistribution

Answer

ImageQuality

Answer

ConfusionMatrix

Question 14

A company is fine-tuning a foundation model using RLHF (Reinforcement Learning from Human Feedback) on SageMaker. They want to reduce memory usage and training time. Which THREE techniques should they consider? (Select THREE.)

Accepted Answer

Use a smaller foundation model (e.g., 7B instead of 70B parameters). LoRA/QLoRA reduces trainable parameters, PPO is the standard RLHF algorithm, and using smaller foundation models reduces memory and compute requirements.

Answer

Use SageMaker Data Parallelism with sharded data

Answer

Use full fine-tuning on a larger instance

Question 15

A data scientist wants to use SageMaker Clarify to analyze bias during training of a binary classification model. Which TWO types of bias metrics can SageMaker Clarify compute? (Select TWO.)

Accepted Answer

Post-training bias metrics (e.g., Difference in Positive Proportions, AD). SageMaker Clarify computes pre-training bias (e.g., class imbalance) and post-training bias (e.g., difference in positive proportions across groups).

Answer

Feature importance

Answer

SHAP values

Answer

Confusion matrix