MLA-C01 ML Model Development Practice Test 6 — 15 Questions

Question 1

A company is using SageMaker Autopilot to automatically build a regression model on a dataset. They want to understand which features are most important for the model's predictions. Which feature of Autopilot can provide this insight?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Autopilot explainability report. SageMaker Autopilot can generate explainability reports that include feature importance, either through SHAP or other methods, depending on the model type.

Answer

Autopilot candidate definition notebook

Answer

Autopilot model leaderboard

Answer

Autopilot data exploration report

Question 2

A machine learning engineer is training a model using SageMaker and wants to set up monitoring to detect if gradients become too large, which could destabilize training. Which SageMaker Debugger built-in rule should they enable?

Accepted Answer

ExplodingGradients. The ExplodingGradients built-in rule in SageMaker Debugger is specifically designed to detect when gradient values become excessively large during training, which can cause numerical instability and prevent convergence. It monitors the gradients tensor and triggers if the ratio of the maximum absolute gradient to the average absolute gradient exceeds a threshold (default 10.0). This directly addresses the engineer's requirement to detect destabilizing gradients.

Answer

DeadRelu

Answer

LossNotDecreasing

Answer

Overfit

Question 3

A data scientist wants to fine-tune a large language model for a question-answering task. They want to reduce memory usage during training by using a low-rank approximation of the weight updates. Which technique should they use?

Accepted Answer

LoRA. LoRA (Low-Rank Adaptation) adds low-rank matrices to model weights, significantly reducing memory footprint while achieving competitive performance. QLoRA adds quantization for further reduction.

Answer

Full fine-tuning

Answer

Instruction tuning

Answer

RLHF

Question 4

A team is building a fraud detection model using SageMaker and wants to detect anomalies in user login events. Which SageMaker built-in algorithm is specifically designed for anomaly detection in event-based data?

Accepted Answer

IP Insights. IP Insights is a SageMaker built-in algorithm specifically designed to learn patterns of IP address usage and detect anomalous behaviour in event-based data such as login events. It embeds IP addresses and entities (e.g., user IDs) into a vector space and flags deviations, making it ideal for fraud detection on login events. The other algorithms serve different purposes.

Answer

Factorisation Machines

Answer

Random Cut Forest

Answer

K-Means

Question 5

A data scientist needs to evaluate a binary classification model. The dataset is highly imbalanced (5% positive class). Which metric is MOST appropriate for assessing model performance?

Accepted Answer

AUC. AUC (Area Under the ROC Curve) evaluates the model's ability to rank positive instances above negatives across all classification thresholds, making it robust to class imbalance because it does not depend on a single threshold or on the majority class dominating the score. With only 5% positives, accuracy and threshold-dependent metrics can be misleading, but AUC remains a reliable discriminator measure.

Answer

Precision

Answer

Accuracy

Answer

Recall

Question 6

A company is using SageMaker to train a model with a custom container. The training script requires a specific version of a Python library that is not included in the default SageMaker containers. How should they provide this library?

Accepted Answer

Extend a SageMaker framework container and install the library using a Dockerfile. Using a custom container (BYOC) allows bundling all dependencies, including specific library versions, into a Docker image that SageMaker can run.

Answer

Use SageMaker Script Mode and specify the library in a requirements.txt

Answer

Use SageMaker's lifecycle configuration to install the library on the training instance

Answer

Use pip install in the training script before model training

Question 7

A data scientist is using SageMaker Autopilot for a regression problem. They want to see which data preprocessing steps Autopilot applied. Which TWO sources can they use to find this information?

Accepted Answer

Candidate definition notebook. The candidate definition notebook (option A) is generated by SageMaker Autopilot for each candidate and contains the full ML pipeline code, including the exact data preprocessing and feature engineering transforms that were applied, so it directly answers the question. The data exploration report (option D) is produced during the Autopilot job's data exploration phase and documents the dataset's characteristics along with the preprocessing and feature engineering steps Autopilot selected, making it another valid source. The model leaderboard (option B) only ranks trained candidates by objective metric and does not describe preprocessing. The Autopilot job description in AWS CloudTrail (option C) records API-level audit events, not the internal preprocessing steps. The explainability report (option E) covers feature attributions and model behavior, not the preprocessing pipeline.

Answer

Model leaderboard

Answer

Autopilot job description in AWS CloudTrail

Answer

Explainability report

Question 8

A data scientist wants to quickly build a binary classification model without writing any code. Which SageMaker feature is MOST suitable?

Accepted Answer

SageMaker Autopilot. SageMaker Autopilot is a no-code AutoML feature that automatically explores data, selects algorithms, tunes hyperparameters, and builds the best model for a given tabular dataset, including binary classification. It requires no model-building code from the user, making it the most suitable choice. The other options are operational/monitoring or labeling tools, not model-building features.

Answer

SageMaker Debugger

Answer

SageMaker Model Monitor

Answer

SageMaker Ground Truth

Question 9

Which SageMaker built-in algorithm is designed for time series forecasting?

Accepted Answer

DeepAR. DeepAR is a SageMaker built-in algorithm specifically designed for time series forecasting. It uses recurrent neural networks (RNNs) to predict future values based on historical data, making it ideal for forecasting tasks such as demand prediction or sales forecasting.

Answer

Linear Learner

Answer

Factorisation Machines

Answer

BlazingText

Question 10

A machine learning engineer wants to reduce training costs by using excess EC2 capacity. Which instance purchasing option should they choose for SageMaker training jobs?

Accepted Answer

Spot Instances. Spot Instances use AWS's spare EC2 capacity and are discounted up to 90% versus On-Demand, making them the correct choice when the workload is interruptible. SageMaker training jobs support Managed Spot Training, which automatically checkpoints and resumes jobs when capacity is reclaimed, so the engineer gets the cost savings without losing training progress. Reserved Instances and Dedicated Instances do not offer the same spot-level discount for interruptible training.

Answer

Reserved Instances

Answer

On-Demand Instances

Answer

Dedicated Instances

Question 11

A team is training a large language model and needs to split the model layers across multiple GPUs due to memory constraints. Which distributed training strategy should they use?

Accepted Answer

Model parallelism. Model parallelism splits the model's layers (or tensors) across multiple GPUs so that each GPU holds only a portion of the model's parameters, which is required when the model is too large to fit in a single GPU's memory. Data parallelism, by contrast, replicates the full model on every GPU and only partitions the training data, so it does not solve a memory-constraint problem.

Answer

Data parallelism

Answer

Hyperparameter tuning

Answer

Autopilot

Question 12

A data scientist is using SageMaker Automatic Model Tuning to optimize hyperparameters for an XGBoost model. They want to maximize AUC. Which search strategy is MOST appropriate for efficient exploration?

Accepted Answer

Bayesian optimization. Bayesian optimization is the most appropriate search strategy for efficiently exploring hyperparameter space because it builds a probabilistic surrogate model of the objective function (AUC) and uses an acquisition function to intelligently select the next hyperparameter combination to evaluate. This makes it far more sample-efficient than grid or random search, which is critical when each training run is expensive. SageMaker Automatic Model Tuning supports Bayesian optimization as its default and recommended strategy.

Answer

Random search

Answer

Grid search

Answer

Hyperband

Question 13

A team is fine-tuning a Hugging Face transformer model on SageMaker. They need to use a custom training script with the Hugging Face Estimator. Which SageMaker feature does this represent?

Accepted Answer

Script mode. Using a custom training script with the Hugging Face Estimator in SageMaker represents Script Mode. Script Mode allows you to bring your own training script and run it within a pre-built framework container, such as Hugging Face, PyTorch, or TensorFlow. This provides flexibility to customize training logic while leveraging SageMaker's managed infrastructure.

Answer

Built-in algorithm

Answer

SageMaker Autopilot

Answer

SageMaker Debugger

Question 14

A machine learning engineer runs a training job and notices the loss is NaN after a few steps. Which SageMaker Debugger rule can help identify this issue?

Accepted Answer

Exploding gradients. The SageMaker Debugger built-in 'ExplodingGradients' rule monitors gradient tensors during training and detects when gradient values grow excessively large, which is a classic cause of NaN loss values. When gradients explode, weight updates become enormous, causing numerical overflow and NaN in the loss. The ExplodingGradients rule flags this condition so the engineer can apply gradient clipping or reduce the learning rate.

Answer

Overfit

Answer

Dead ReLU

Answer

Class imbalance

Question 15

A company needs to detect bias in a pre-trained model before deployment. They want to compute metrics like disparate impact and equal opportunity difference. Which AWS service should they use?

Accepted Answer

SageMaker Clarify. SageMaker Clarify is purpose-built to detect bias in datasets and models, computing pre-training and post-training bias metrics such as disparate impact (DI) and equal opportunity difference (EOD). It integrates with SageMaker training and hosting, and can also generate explainability reports using SHAP values. Because the question asks specifically for bias metrics like DI and EOD, Clarify is the correct service.

Answer

Amazon Rekognition

Answer

SageMaker Model Monitor

Answer

SageMaker Debugger