MLA-C01 ML Model Development Practice Test 4 — 15 Questions

Question 1

A machine learning engineer is using SageMaker Autopilot for AutoML. Which TWO outputs does Autopilot produce?

Accepted Answer

An ensemble of candidate models

Answer

A hyperparameter tuning job summary

Answer

A data labeling pipeline

Answer

A single optimal model

Question 2

A data scientist wants to bring a custom PyTorch model to SageMaker. Which THREE methods are valid?

Accepted Answer

Use Script mode with the PyTorch Estimator

Answer

Use SageMaker Autopilot

Answer

Use the built-in Image Classification algorithm

Question 3

A company is fine-tuning a large language model using reinforcement learning from human feedback (RLHF). Which THREE components are typically required?

Accepted Answer

A reference model

Answer

A discriminative classifier

Answer

A value function

Question 4

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Answer

Fine-tune a base LLM on the policy documents monthly

Answer

Train a custom model from scratch on the policy documents each month

Question 5

A data scientist wants to train a binary classification model using Amazon SageMaker. The dataset has 10,000 rows and 50 features. Which SageMaker built-in algorithm is MOST appropriate for this task?

Accepted Answer

XGBoost. XGBoost is a popular algorithm for classification and regression tasks. Linear Learner is more suited for linear models, K-Means is for clustering, and DeepAR is for time series forecasting.

Answer

DeepAR

Answer

K-Means

Answer

Linear Learner

Question 6

A company is training a large computer vision model using SageMaker. The training dataset is 500 GB and the model has 1 billion parameters. The team needs to minimize training time. Which distributed training strategy should they use?

Accepted Answer

Model parallelism. Model parallelism splits the model layers across multiple GPUs, which is necessary when the model is too large to fit on a single GPU. Data parallelism replicates the model on each GPU and splits the data, but is limited by the memory of a single GPU.

Answer

Pipeline parallelism

Answer

Sharded data parallelism

Answer

Data parallelism

Question 7

A machine learning engineer is using SageMaker Automatic Model Tuning to optimize hyperparameters for a regression model. The objective metric is RMSE. The training job is costly, and the engineer wants to find a good configuration quickly. Which tuning strategy should they use?

Accepted Answer

Bayesian optimization. Bayesian optimization builds a probabilistic model of the objective function and selects hyperparameters to try next based on past results, making it more efficient than random search. Hyperband is a bandit-based approach that may be faster but can be less stable.

Answer

Hyperband

Answer

Random search

Answer

Grid search

Question 8

A team is training a PyTorch model using SageMaker. They have a custom training script that requires specific Python packages not included in the SageMaker default PyTorch container. Which approach should they use?

Accepted Answer

Use the built-in PyTorch estimator and specify a requirements.txt in the source directory. Using a SageMaker PyTorch estimator with a requirements.txt file allows installing additional packages on top of the official container. This is simpler than building a custom container.

Answer

Build a custom Docker container from scratch and push it to Amazon ECR

Answer

Use the SageMaker XGBoost estimator and modify the script to use PyTorch

Answer

Use SageMaker Autopilot to automatically handle dependencies

Question 9

A data scientist wants to track hyperparameters, metrics, and artifacts for multiple training runs in SageMaker. They need to compare runs and identify the best performing model. Which SageMaker feature should they use?

Accepted Answer

SageMaker Experiments. SageMaker Experiments allows tracking and comparing runs, including hyperparameters, metrics, and artifacts.

Answer

SageMaker Model Monitor

Answer

SageMaker Debugger

Answer

SageMaker Autopilot

Question 10

A financial services firm is training a fraud detection model using SageMaker. The dataset is highly imbalanced (0.1% fraudulent transactions). The model currently achieves 99.9% accuracy but only catches 5% of fraud cases. Which metric should the team prioritize to evaluate model performance?

Accepted Answer

Recall. Recall (true positive rate) measures the proportion of actual positives correctly identified. For fraud detection, catching fraud is critical; accuracy is misleading due to class imbalance.

Answer

Accuracy

Answer

Precision

Answer

F1-score

Question 11

A company is fine-tuning a large language model using LoRA with a Hugging Face estimator in SageMaker. They want to reduce memory usage during training. Which instance type is most cost-effective for this workload?

Accepted Answer

ml.g5.xlarge. LoRA reduces the number of trainable parameters, allowing training on smaller GPUs. ml.g5 instances are optimized for machine learning inference and training with a good price-performance for fine-tuning.

Answer

ml.p4d.24xlarge

Answer

ml.c5.2xlarge

Answer

ml.trn1.2xlarge

Question 12

A data scientist wants to use SageMaker Autopilot to automatically build a regression model. The dataset contains 200 features and 50,000 rows. Which output does SageMaker Autopilot provide?

Accepted Answer

A leaderboard of candidate models with metrics and explainability reports. SageMaker Autopilot automatically explores various algorithms and preprocessing steps, then provides a leaderboard of candidate models with metrics.

Answer

Only the best model without any metrics

Answer

A single optimal model with no further tuning

Answer

A Python script for manual training

Question 13

A company is using SageMaker Debugger to monitor a training job for a deep learning model. They want to detect when gradients become extremely large, which may cause training instability. Which built-in rule should they use?

Accepted Answer

ExplodingGradients. The ExplodingGradients rule monitors gradient norms and raises an alert if they exceed a threshold.

Answer

DeadRelu

Answer

VanishingGradients

Answer

Overfit

Question 14

A team is fine-tuning a foundation model using reinforcement learning from human feedback (RLHF) on SageMaker. They have a dataset of human preferences. Which SageMaker capability is most suitable for the reward model training step?

Accepted Answer

SageMaker Training with a custom PyTorch container. RLHF typically involves training a reward model on human preference data. SageMaker can be used to train any custom model, including a reward model, using its training jobs with a PyTorch or TensorFlow estimator.

Answer

SageMaker JumpStart

Answer

SageMaker Ground Truth

Answer

SageMaker Autopilot

Question 15

A company is using SageMaker to train a model for image classification. The training dataset contains 100,000 labeled images. The team wants to use a pre-trained model to reduce training time. Which SageMaker feature should they use?

Accepted Answer

SageMaker JumpStart. SageMaker JumpStart provides pre-trained models that can be fine-tuned on custom datasets, reducing training time and data requirements.

Answer

SageMaker Debugger

Answer

SageMaker Model Monitor

Answer

SageMaker built-in Image Classification algorithm