MLA-C01 ML Model Development — Timed 15 Questions

Question 1

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. RAG allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining.

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Answer

Fine-tune a base LLM on the policy documents monthly

Answer

Train a custom model from scratch on the policy documents each month

Question 2

A data scientist is using SageMaker built-in XGBoost algorithm for a binary classification task. Which objective metric is MOST appropriate for SageMaker Automatic Model Tuning to maximize?

Accepted Answer

validation:auc

Answer

validation:mae

Answer

validation:rmse

Answer

validation:ndcg

Question 3

A team is training a large language model using SageMaker with multiple GPUs. They need to reduce training time by splitting the model across devices due to memory constraints. Which distributed training strategy should they use?

Accepted Answer

Model parallelism

Answer

SageMaker Distributed Data Parallel (SMDDP)

Answer

Data parallelism

Answer

SageMaker Distributed Model Parallel (SMDMP)

Question 4

A machine learning engineer is using SageMaker Debugger to monitor training jobs. They want to capture tensors every 100 steps but only for the first 500 steps. Which configuration should they set in the Debugger hook?

Accepted Answer

collection_configs with save_interval=100 and end_step=500

Answer

collection_configs with save_interval=500 and end_step=100

Answer

collection_configs with start_step=100 and end_step=500

Answer

Use SageMaker Debugger rules to filter steps

Question 5

A company wants to use SageMaker Autopilot for a regression problem. They require an explainability report that shows feature importance globally. Which Autopilot feature should they enable?

Accepted Answer

Explainability report generation

Answer

AutoML candidate generation

Answer

Ensembling mode

Answer

Hyperparameter optimization

Question 6

A team is fine-tuning a foundation model using LoRA in SageMaker. They want to reduce memory usage during training. Which instance type is optimized for cost-effective fine-tuning with LoRA?

Accepted Answer

ml.g5.2xlarge

Answer

ml.p3.2xlarge

Answer

ml.c5.2xlarge

Answer

ml.trn1.2xlarge

Question 7

A data scientist uses SageMaker Experiments to track hyperparameters and metrics. Which component is used to organize related trials?

Accepted Answer

Experiment

Answer

Artifact

Answer

Trial component

Answer

Trial

Question 8

A company uses SageMaker Clarify to detect bias during training. They want to ensure that the trained model does not rely on a sensitive attribute like gender. Which Clarify feature should they configure?

Accepted Answer

Clarify bias config with post-training bias metrics

Answer

Clarify with SageMaker Model Monitor

Answer

SHAP analysis

Answer

Clarify processing job with pre-training bias metrics

Answer

Bias report generation

Question 9

A team wants to use a custom PyTorch training script in SageMaker. They need to install additional Python packages not included in the base PyTorch container. Which approach should they take?

Accepted Answer

Use the SageMaker PyTorch estimator with a requirements.txt file

Answer

Use SageMaker Script Mode with a custom Dockerfile

Answer

Build a custom container with Docker

Answer

Install packages using a lifecycle configuration

Question 10

A practitioner is using SageMaker Automatic Model Tuning with Hyperband strategy. They want to stop underperforming trials early to save compute. Which Hyperband parameter controls the aggressiveness of early stopping?

Accepted Answer

early_stopping_type

Answer

strategy

Answer

max_jobs

Answer

max_parallel_jobs

Question 11

A company needs to perform time-series forecasting on historical sales data. Which SageMaker built-in algorithm is BEST suited for this task?

Accepted Answer

DeepAR

Answer

BlazingText

Answer

Linear Learner

Answer

XGBoost

Question 12

A data scientist is training an object detection model using SageMaker built-in Object Detection algorithm. They want to visualize the bounding boxes on validation images after training. Which approach should they use?

Accepted Answer

Write a custom inference script that saves images with bounding boxes

Answer

Use SageMaker Debugger to capture output tensors

Answer

Enable SageMaker Model Monitor

Answer

Use SageMaker Clarify

Question 13

A machine learning engineer wants to reduce costs for hyperparameter tuning jobs that run for several hours. The jobs are fault-tolerant and can be interrupted. Which TWO actions should they take? (Select TWO.)

Accepted Answer

Enable SageMaker Managed Spot Training

Answer

Use on-demand instances for reliability

Answer

Use ml.c5 instances instead of ml.p3

Answer

Increase the number of parallel training jobs

Question 14

A data scientist is evaluating a binary classification model. They have the confusion matrix and want to assess the model's performance comprehensively. Which THREE metrics should they consider? (Select THREE.)

Accepted Answer

Precision

Answer

RMSE

Answer

R²

Question 15

A team is fine-tuning a large language model using reinforcement learning from human feedback (RLHF) in SageMaker. Which THREE components are essential for the RLHF pipeline? (Select THREE.)

Accepted Answer

Policy network (the LLM being fine-tuned)

Answer

Feature store

Answer

Hyperparameter tuner