MLA-C01 ML Model Development Practice Test 2 — 15 Questions

Question 1

A team is training a large model on SageMaker using the SageMaker distributed training library with model parallelism. They need to choose the most cost-effective instance type. Which instance family offers the best balance of performance and cost for large model training?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

ml.trn1

Answer

ml.g4dn

Answer

ml.p3

Answer

ml.c5

Question 2

A data scientist wants to bring a custom PyTorch model to SageMaker. Which THREE methods are valid?

Accepted Answer

Use Script mode with the PyTorch Estimator. Option C is correct because SageMaker's Script mode with the PyTorch Estimator lets you supply your own training script (entry_point) while SageMaker manages the PyTorch framework container, which is the standard way to bring a custom PyTorch model. Option D is correct because the Bring Your Own Container (BYOC) approach lets you package any custom PyTorch code and dependencies into a Docker image that satisfies the SageMaker training/inference container contract, giving full control over the environment. Option E is correct because using the PyTorch Estimator with a script is functionally the same valid pattern as Script mode — you pass your custom PyTorch training script to the estimator, which runs it in the managed PyTorch container. Option A is not correct because SageMaker Autopilot is an automated machine learning service that explores built-in algorithms and feature engineering for tabular data; it does not accept or train arbitrary custom PyTorch models. Option B is not correct because the built-in Image Classification algorithm is a pre-built, fixed algorithm with its own training logic and hyperparameters, so it cannot run a user's custom PyTorch model code.

Answer

Use SageMaker Autopilot

Answer

Use the built-in Image Classification algorithm

Question 3

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Answer

Fine-tune a base LLM on the policy documents monthly

Answer

Train a custom model from scratch on the policy documents each month

Question 4

A data scientist wants to train a binary classification model using Amazon SageMaker. The dataset has 10,000 rows and 50 features. Which SageMaker built-in algorithm is MOST appropriate for this task?

Accepted Answer

XGBoost. XGBoost is a popular algorithm for classification and regression tasks. Linear Learner is more suited for linear models, K-Means is for clustering, and DeepAR is for time series forecasting.

Answer

DeepAR

Answer

K-Means

Answer

Linear Learner

Question 5

A company is training a large computer vision model using SageMaker. The training dataset is 500 GB and the model has 1 billion parameters. The team needs to minimize training time. Which distributed training strategy should they use?

Accepted Answer

Model parallelism. Model parallelism splits the model layers across multiple GPUs, which is necessary when the model is too large to fit on a single GPU. Data parallelism replicates the model on each GPU and splits the data, but is limited by the memory of a single GPU.

Answer

Pipeline parallelism

Answer

Sharded data parallelism

Answer

Data parallelism

Question 6

A company is fine-tuning a large language model using LoRA with a Hugging Face estimator in SageMaker. They want to reduce memory usage during training. Which instance type is most cost-effective for this workload?

Accepted Answer

ml.g5.xlarge. LoRA fine-tuning freezes the base model and trains small low-rank adapter matrices, drastically reducing memory for optimizer states and gradients. An ml.g5.xlarge with an NVIDIA A10G GPU (24 GB) provides sufficient GPU memory and is the most cost-effective choice among the options for this workload.

Answer

ml.p4d.24xlarge

Answer

ml.c5.2xlarge

Answer

ml.trn1.2xlarge

Question 7

A data scientist is training an XGBoost model on a large dataset using a SageMaker Training Job. They want to minimize costs without sacrificing model performance. Which instance type and training strategy should they choose?

Accepted Answer

Use a single ml.g4dn.xlarge Spot instance with no distributed training. A single ml.g4dn.xlarge Spot instance with no distributed training minimizes cost because Spot pricing is significantly cheaper than On-Demand and XGBoost on a moderate dataset does not require multi-GPU distribution. The g4dn family provides GPU acceleration at a lower price point than p3 or trn1, and avoiding distributed training eliminates inter-node communication overhead. This combination balances cost and performance for the stated workload.

Answer

Use a single ml.m5.large On-Demand instance with model parallelism

Answer

Use multiple ml.trn1.2xlarge On-Demand instances with data parallelism

Answer

Use a single ml.p3.2xlarge On-Demand instance with data parallelism

Question 8

A data scientist uses SageMaker Automatic Model Tuning (AMT) with Bayesian optimization to tune an XGBoost model. The objective metric is validation:auc, but the tuning job converges to a plateau early. Which action is MOST effective to improve exploration?

Accepted Answer

Increase the exploration_weight parameter in the tuning configuration. In SageMaker AMT Bayesian optimization, the exploration_weight parameter controls the trade-off between exploration (trying new hyperparameter regions) and exploitation (refining known good regions). When the tuning job plateaus early, it is over-exploiting; increasing exploration_weight pushes the optimizer to sample more diverse configurations and escape the local optimum. This directly addresses the early convergence described.

Answer

Increase the number of max parallel jobs

Answer

Decrease the number of hyperparameters being tuned

Answer

Switch the tuning strategy from Bayesian to Random Search

Question 9

A data scientist is training a linear learner model using SageMaker and notices that the loss is not decreasing. They suspect the issue is exploding gradients. Which SageMaker Debugger rule should they enable to monitor this?

Accepted Answer

ExplodingGradients. The ExplodingGradients rule tracks gradient values and alerts if they exceed a threshold, which is the correct detection for exploding gradients.

Answer

LossNotDecreasing

Answer

VanishingGradients

Answer

Overfit

Question 10

A company is fine-tuning a foundation model using RLHF (Reinforcement Learning from Human Feedback) on SageMaker. They want to reduce memory usage and training time. Which THREE techniques should they consider? (Select THREE.)

Accepted Answer

Use a smaller foundation model (e.g., 7B instead of 70B parameters). Option A is correct because selecting a smaller foundation model such as a 7B-parameter model instead of a 70B-parameter model directly reduces the GPU memory footprint (weights, gradients, optimizer states, and activations) and shortens each training step, which lowers overall training time. Option B is correct because PPO is the standard, stable policy-gradient algorithm used for the RLHF reward-optimization stage; its clipped objective limits destructive policy updates, avoiding the instability and extra rollouts/retries that less stable RL methods would cause, thereby saving compute and time. Option E is correct because LoRA and QLoRA freeze the base weights and train only small low-rank adapter matrices (QLoRA additionally quantizes the base model to 4-bit), which drastically cuts trainable parameters, optimizer memory, and checkpoint size, reducing both memory usage and training time. Option C is not appropriate here because SageMaker data parallelism shards data across replicas but still replicates the full model on each GPU, so it does not reduce per-device model memory for a large foundation model. Option D is incorrect because full fine-tuning on a larger instance increases memory consumption and cost rather than reducing them, and it does not address the goal of faster, lighter RLHF training.

Answer

Use SageMaker Data Parallelism with sharded data

Answer

Use full fine-tuning on a larger instance

Question 11

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. Retrieval-Augmented Generation (RAG) with a vector store allows the chatbot to retrieve relevant policy documents dynamically without retraining the model. When documents are updated monthly, the vector store can be re-indexed, and the LLM uses the retrieved context to generate accurate answers, avoiding the cost and delay of fine-tuning or retraining.

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Answer

Fine-tune a base LLM on the policy documents monthly

Answer

Train a custom model from scratch on the policy documents each month

Question 12

A data scientist is training an XGBoost model on a large tabular dataset using SageMaker. The training job is taking too long. The scientist wants to reduce training time while maintaining model quality. Which action should the scientist take?

Accepted Answer

Use SageMaker distributed data parallelism across multiple instances. Using SageMaker's managed spot training can significantly reduce cost, but it may cause interruptions. The best approach to reduce training time is to use distributed data parallelism with multiple instances. Increasing instance type can also speed up training, but distributed training is more scalable. Using Hyperband is for hyperparameter tuning, not for reducing training time directly. Converting to a different algorithm is not necessary.

Answer

Enable SageMaker managed spot training

Answer

Switch to Hyperband for hyperparameter tuning

Answer

Convert the XGBoost model to a Linear Learner model

Question 13

A company is training a large Transformer model on SageMaker and wants to use model parallelism to fit the model into memory. The model has 10 billion parameters. Which instance type is MOST cost-effective for this task while supporting SageMaker's model parallelism?

Accepted Answer

ml.p4d.24xlarge. The ml.p4d.24xlarge instances are optimized for large-scale distributed training with high memory and support SageMaker's model parallelism. ml.trn1 instances are designed for training with AWS Trainium, but they use a different chip architecture and may require specific SDKs. ml.g4dn instances are for inference and light training. ml.c5 instances are compute-optimized but lack GPU memory for large models.

Answer

ml.trn1.32xlarge

Answer

ml.c5.18xlarge

Answer

ml.g4dn.12xlarge

Question 14

A data scientist is using SageMaker Autopilot to automatically build a binary classification model. The dataset is imbalanced. Which action will Autopilot take by default to address class imbalance?

Accepted Answer

Use class balancing weights. SageMaker Autopilot, by default, uses class balancing weights to address class imbalance in binary classification problems. This means it automatically adjusts the loss function to give more weight to the minority class, helping the model learn from imbalanced data without requiring manual resampling. This is a built-in feature of Autopilot's automatic model tuning.

Answer

Perform random undersampling of the majority class

Answer

Ignore the imbalance and proceed with raw data

Answer

Apply SMOTE oversampling

Question 15

An ML engineer is debugging a training job that is consistently failing due to an out-of-memory error. The engineer is using SageMaker's built-in XGBoost algorithm. Which Debugger rule can help identify the issue?

Accepted Answer

Exploding gradients. The 'Exploding gradients' rule detects when gradients become too large, which is a common cause of training instability but not necessarily OOM. The 'Overfit' rule detects overfitting. The 'Dead relu' rule is for ReLU activation. None of these directly address OOM. However, Debugger does not have a specific OOM rule; instead, the engineer should monitor memory utilization via CloudWatch or adjust instance type. Among the options, 'Exploding gradients' is the most relevant because large gradients can lead to memory spikes.

Answer

Overfit

Answer

Dead relu

Answer

OOM rule