MLA-C01 ML Model Development — 30 Questions

Question 1

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. RAG allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining.

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Answer

Fine-tune a base LLM on the policy documents monthly

Answer

Train a custom model from scratch on the policy documents each month

Question 2

A data scientist is using SageMaker built-in XGBoost algorithm for a binary classification task. Which objective metric is MOST appropriate for SageMaker Automatic Model Tuning to maximize?

Accepted Answer

validation:auc

Answer

validation:mae

Answer

validation:rmse

Answer

validation:ndcg

Question 3

A team is training a large language model using SageMaker with multiple GPUs. They need to reduce training time by splitting the model across devices due to memory constraints. Which distributed training strategy should they use?

Accepted Answer

Model parallelism

Answer

SageMaker Distributed Data Parallel (SMDDP)

Answer

Data parallelism

Answer

SageMaker Distributed Model Parallel (SMDMP)

Question 4

A machine learning engineer is using SageMaker Debugger to monitor training jobs. They want to capture tensors every 100 steps but only for the first 500 steps. Which configuration should they set in the Debugger hook?

Accepted Answer

collection_configs with save_interval=100 and end_step=500

Answer

collection_configs with save_interval=500 and end_step=100

Answer

collection_configs with start_step=100 and end_step=500

Answer

Use SageMaker Debugger rules to filter steps

Question 5

A company wants to use SageMaker Autopilot for a regression problem. They require an explainability report that shows feature importance globally. Which Autopilot feature should they enable?

Accepted Answer

Explainability report generation

Answer

AutoML candidate generation

Answer

Ensembling mode

Answer

Hyperparameter optimization

Question 6

A team is fine-tuning a foundation model using LoRA in SageMaker. They want to reduce memory usage during training. Which instance type is optimized for cost-effective fine-tuning with LoRA?

Accepted Answer

ml.g5.2xlarge

Answer

ml.p3.2xlarge

Answer

ml.c5.2xlarge

Answer

ml.trn1.2xlarge

Question 7

A data scientist uses SageMaker Experiments to track hyperparameters and metrics. Which component is used to organize related trials?

Accepted Answer

Experiment

Answer

Artifact

Answer

Trial component

Answer

Trial

Question 8

A company uses SageMaker Clarify to detect bias during training. They want to ensure that the trained model does not rely on a sensitive attribute like gender. Which Clarify feature should they configure?

Accepted Answer

Clarify bias config with post-training bias metrics

Answer

Clarify with SageMaker Model Monitor

Answer

SHAP analysis

Answer

Clarify processing job with pre-training bias metrics

Answer

Bias report generation

Question 9

A team wants to use a custom PyTorch training script in SageMaker. They need to install additional Python packages not included in the base PyTorch container. Which approach should they take?

Accepted Answer

Use the SageMaker PyTorch estimator with a requirements.txt file

Answer

Use SageMaker Script Mode with a custom Dockerfile

Answer

Build a custom container with Docker

Answer

Install packages using a lifecycle configuration

Question 10

A practitioner is using SageMaker Automatic Model Tuning with Hyperband strategy. They want to stop underperforming trials early to save compute. Which Hyperband parameter controls the aggressiveness of early stopping?

Accepted Answer

early_stopping_type

Answer

strategy

Answer

max_jobs

Answer

max_parallel_jobs

Question 11

A company needs to perform time-series forecasting on historical sales data. Which SageMaker built-in algorithm is BEST suited for this task?

Accepted Answer

DeepAR

Answer

BlazingText

Answer

Linear Learner

Answer

XGBoost

Question 12

A data scientist is training an object detection model using SageMaker built-in Object Detection algorithm. They want to visualize the bounding boxes on validation images after training. Which approach should they use?

Accepted Answer

Write a custom inference script that saves images with bounding boxes

Answer

Use SageMaker Debugger to capture output tensors

Answer

Enable SageMaker Model Monitor

Answer

Use SageMaker Clarify

Question 13

A machine learning engineer wants to reduce costs for hyperparameter tuning jobs that run for several hours. The jobs are fault-tolerant and can be interrupted. Which TWO actions should they take? (Select TWO.)

Accepted Answer

Enable SageMaker Managed Spot Training

Answer

Use on-demand instances for reliability

Answer

Use ml.c5 instances instead of ml.p3

Answer

Increase the number of parallel training jobs

Question 14

A data scientist is evaluating a binary classification model. They have the confusion matrix and want to assess the model's performance comprehensively. Which THREE metrics should they consider? (Select THREE.)

Accepted Answer

Precision

Answer

RMSE

Answer

R²

Question 15

A team is fine-tuning a large language model using reinforcement learning from human feedback (RLHF) in SageMaker. Which THREE components are essential for the RLHF pipeline? (Select THREE.)

Accepted Answer

Policy network (the LLM being fine-tuned)

Answer

Feature store

Answer

Hyperparameter tuner

Question 16

A data scientist wants to train a binary classification model using Amazon SageMaker with a built-in algorithm that performs well on tabular data. Which algorithm should they choose?

Accepted Answer

XGBoost. XGBoost is a popular built-in algorithm in SageMaker for classification and regression on tabular data. Linear Learner is also for tabular data but XGBoost often performs better for complex patterns.

Answer

Image Classification

Answer

DeepAR

Answer

BlazingText

Question 17

A machine learning engineer needs to reduce costs when training a large model on SageMaker. They are willing to accept potential interruptions and have checkpointing enabled. Which instance purchasing option should they use?

Accepted Answer

Spot instances. Spot instances offer significant cost savings (up to 60-90%) compared to on-demand, but can be reclaimed by AWS with a 2-minute notice. Checkpointing allows resuming training from the last saved state, making spot instances suitable.

Answer

Reserved instances

Answer

Dedicated hosts

Answer

On-demand instances

Question 18

A data scientist is using SageMaker Automatic Model Tuning to find the best hyperparameters for a model. They want to reduce the total tuning time for a given number of training jobs. Which tuning strategy should they choose?

Accepted Answer

Hyperband. Hyperband is an early stopping strategy that allocates resources to promising configurations and stops poor performers early, reducing total tuning time compared to random search or Bayesian optimization without early stopping.

Answer

Grid search

Answer

Random search

Answer

Bayesian optimization

Question 19

A team is training a large language model on SageMaker using PyTorch with data parallelism. The model is too large to fit on a single GPU. Which distributed training strategy should they use to split the model across multiple GPUs?

Accepted Answer

Model parallelism. Model parallelism splits the model itself across devices, which is necessary when the model is too large for one GPU. SageMaker's model parallelism library supports this.

Answer

Tensor parallelism

Answer

Data parallelism

Answer

Pipeline parallelism

Question 20

A data scientist is using SageMaker Experiments to track multiple training runs. They want to compare different hyperparameter configurations and visualize the impact on model accuracy. What should they use to track hyperparameters?

Accepted Answer

SageMaker Experiments. SageMaker Experiments allows you to log hyperparameters as parameters. They can be viewed and compared across runs in the SageMaker Studio UI.

Answer

SageMaker Debugger

Answer

SageMaker Autopilot

Answer

SageMaker Model Monitor