MLA-C01 ML Model Development — Timed 15 Questions

Question 1

A team is training a large language model using SageMaker with multiple GPUs. They need to reduce training time by splitting the model across devices due to memory constraints. Which distributed training strategy should they use?

15 minute time limit — choose an answer to begin.

Accepted Answer

Model parallelism. Model parallelism is the correct strategy because it splits the model itself across multiple devices, which directly addresses the memory constraint of a large language model that cannot fit on a single GPU. When a model is too large for one device's memory, model parallelism partitions the model's layers or parameters across GPUs, allowing training to proceed. This reduces per-device memory usage and enables training of models that would otherwise be impossible, thereby reducing training time by leveraging multiple devices.

Answer

SageMaker Distributed Data Parallel (SMDDP)

Answer

Data parallelism

Answer

SageMaker Distributed Model Parallel (SMDMP)

Question 2

A machine learning engineer is using SageMaker Debugger to monitor training jobs. They want to capture tensors every 100 steps but only for the first 500 steps. Which configuration should they set in the Debugger hook?

Accepted Answer

collection_configs with save_interval=100 and end_step=500. SageMaker Debugger's collection_configs accept 'save_interval' (capture every N steps) and 'end_step' (stop capturing after this step). To capture every 100 steps for the first 500 steps, the correct configuration is save_interval=100 and end_step=500. This produces tensor captures at steps 100, 200, 300, 400, and 500, matching the requirement exactly.

Answer

collection_configs with save_interval=500 and end_step=100

Answer

collection_configs with start_step=100 and end_step=500

Answer

Use SageMaker Debugger rules to filter steps

Question 3

A team is fine-tuning a foundation model using LoRA in SageMaker. They want to reduce memory usage during training. Which instance type is optimized for cost-effective fine-tuning with LoRA?

Accepted Answer

ml.g5.2xlarge. ml.g5.2xlarge is correct because it is equipped with NVIDIA A10G GPUs, which provide a good balance of memory and compute for parameter-efficient fine-tuning methods like LoRA. LoRA reduces the number of trainable parameters, so a mid-range GPU instance is sufficient and cost-effective, avoiding the higher cost of larger instances. This instance type is specifically optimized for graphics-intensive and ML workloads, making it ideal for LoRA fine-tuning.

Answer

ml.p3.2xlarge

Answer

ml.c5.2xlarge

Answer

ml.trn1.2xlarge

Question 4

A data scientist uses SageMaker Experiments to track hyperparameters and metrics. Which component is used to organize related trials?

Accepted Answer

Experiment

Answer

Artifact

Answer

Trial component

Answer

Trial

Question 5

A company uses SageMaker Clarify to detect bias during training. They want to ensure that the trained model does not rely on a sensitive attribute like gender. Which Clarify feature should they configure?

Accepted Answer

Clarify bias config with post-training bias metrics

Answer

Clarify with SageMaker Model Monitor

Answer

SHAP analysis

Answer

Clarify processing job with pre-training bias metrics

Answer

Bias report generation

Question 6

A practitioner is using SageMaker Automatic Model Tuning with Hyperband strategy. They want to stop underperforming trials early to save compute. Which Hyperband parameter controls the aggressiveness of early stopping?

Accepted Answer

early_stopping_type. The `early_stopping_type` parameter in SageMaker Automatic Model Tuning controls whether early stopping is enabled ('Auto') or disabled ('Off'). When using the Hyperband strategy, setting `early_stopping_type` to 'Auto' allows SageMaker to stop underperforming trials early based on the Hyperband algorithm's resource allocation and promotion decisions. This directly governs the aggressiveness of early stopping by enabling the mechanism that terminates poor-performing trials.

Answer

strategy

Answer

max_jobs

Answer

max_parallel_jobs

Question 7

A machine learning engineer wants to reduce costs for hyperparameter tuning jobs that run for several hours. The jobs are fault-tolerant and can be interrupted. Which TWO actions should they take? (Select TWO.)

Accepted Answer

Enable SageMaker Managed Spot Training. Option B is correct because SageMaker Managed Spot Training lets training jobs run on spare EC2 capacity at up to a 90% discount, and it automatically manages spot interruptions by checkpointing and resuming jobs, which suits fault-tolerant, interruptible hyperparameter tuning workloads. Option C is correct because using spot instances directly for the training jobs leverages the same discounted spare capacity, and since the jobs are fault-tolerant and can be interrupted, the risk of spot reclamation is acceptable in exchange for lower cost. Option A is incorrect because on-demand instances cost more and are unnecessary for jobs that can tolerate interruption. Option D is incorrect because changing instance families addresses compute type or GPU needs, not cost reduction through interruptible capacity. Option E is incorrect because increasing parallel jobs can raise total cost and does not by itself reduce the price of the compute used.

Answer

Use on-demand instances for reliability

Answer

Use ml.c5 instances instead of ml.p3

Answer

Increase the number of parallel training jobs

Question 8

A team is fine-tuning a large language model using reinforcement learning from human feedback (RLHF) in SageMaker. Which THREE components are essential for the RLHF pipeline? (Select THREE.)

Accepted Answer

Policy network (the LLM being fine-tuned). The standard RLHF pipeline for fine-tuning an LLM requires three components: a policy network (the LLM being fine-tuned), a reward model (provides human feedback), and a reference model (used for KL divergence to prevent reward hacking). Among the given options, only the Policy network (A) and Reward model (E) are essential. The Value network (B) is not a standard component in RLHF; it is used in some reinforcement learning algorithms like PPO but not as a core part of RLHF for LLMs. The Feature store (C) and Hyperparameter tuner (D) are useful but not essential to the RLHF pipeline. Since only two of the three essential components are listed, the correct selections are A and E.

Answer

Value network

Answer

Feature store

Answer

Hyperparameter tuner

Question 9

A team is training a large language model using SageMaker with multiple GPUs. They need to reduce training time by splitting the model across devices due to memory constraints. Which distributed training strategy should they use?

15 minute time limit — choose an answer to begin.

Accepted Answer

Model parallelism. Model parallelism splits the model itself across devices, which is necessary when the model is too large for one GPU. SageMaker's model parallelism library supports this.

Answer

Tensor parallelism

Answer

Data parallelism

Answer

Pipeline parallelism

Question 10

A machine learning engineer is using SageMaker Debugger to detect if a neural network has dead ReLU units during training. Which built-in rule should they enable?

Accepted Answer

DeadRelu. SageMaker Debugger's built-in DeadRelu rule is specifically designed to detect dead ReLU activations during training by monitoring the fraction of ReLU units whose activations remain at zero across training steps. When a ReLU neuron's weights become negative enough that it never activates again, its gradient is permanently zero and it stops learning — this is exactly the 'dying ReLU' pathology the rule targets. The rule emits a training job issue when the percentage of dead units exceeds a configurable threshold, allowing engineers to intervene (e.g., lower learning rate, switch to LeakyReLU).

Answer

Overfit

Answer

ExplodingGradients

Answer

LossNotDecreasing

Question 11

A company is using SageMaker to train a large model using data parallelism with the SageMaker distributed data parallelism library. They notice that the training throughput is not scaling linearly with the number of GPUs. Which THREE factors could be causing this?

Accepted Answer

I/O bottleneck from reading data from Amazon S3. Option A is correct because in SageMaker distributed data parallelism, each GPU replica must continuously fetch training data, so an I/O bottleneck from reading data from Amazon S3 (or slow input pipelines) starves the GPUs and prevents throughput from scaling linearly. Option D is correct because the SageMaker distributed data parallelism library relies on loss scaling to keep gradients in a numerically stable range for FP16 training; an inefficient or poorly tuned loss scaling strategy causes overflow/underflow, skipped steps, and reduced effective throughput. Option E is correct because data parallelism requires AllReduce-style gradient synchronization across all GPUs, and as the number of GPUs grows, the communication overhead from gradient synchronization grows and can dominate, breaking linear scaling. Option B is not correct because SageMaker distributed training requires homogeneous clusters, and using different instance types is not a supported configuration that would cause this scaling issue. Option C is not correct because a model too large for the GPUs would cause out-of-memory failures rather than a gradual loss of throughput scaling.

Answer

Using different instance types across the cluster

Answer

Model size too large for the GPUs

Question 12

A machine learning engineer wants to use SageMaker Clarify to analyze bias in their training data and model predictions. They want to detect bias before training. Which TWO types of analysis can SageMaker Clarify perform on the data?

Accepted Answer

Pre-training bias metrics (e.g., class imbalance, feature skew). SageMaker Clarify can compute pre-training bias metrics such as class imbalance and feature skew directly on the training data before any model is trained. This is the only type of analysis among the options that is performed before training. Feature importance (SHAP values) and explainability reports require a trained model, so they are post-training analyses. Model monitoring for data drift is a separate feature of SageMaker. Post-training bias metrics also require model predictions. Therefore, only option A qualifies as a pre-training analysis.

Answer

Feature importance (SHAP values) on the dataset

Answer

Model monitoring for data drift

Answer

Explainability report for the model

Answer

Post-training bias metrics (e.g., difference in accuracy across groups)

Question 13

A company uses SageMaker Experiments to track training runs. They want to compare different hyperparameter configurations and identify the best run. Which SageMaker Experiments component should they use to organize related runs?

Accepted Answer

Experiment + Trial

Answer

Trial

Answer

Experiment

Answer

Trial Component

Question 14

A fraud detection model is being trained on imbalanced data. The team wants to ensure the model's precision is optimized. Which objective metric should be used in automatic model tuning?

Accepted Answer

Precision

Answer

F1

Answer

AUC

Answer

Recall

Question 15

A team is fine-tuning a foundation model using LoRA for a text summarization task. They want to reduce memory footprint during training. Which technique should they combine with LoRA?

Accepted Answer

QLoRA. QLoRA (Quantized LoRA) extends LoRA by quantizing the frozen base model weights to 4-bit precision while keeping the LoRA adapters in higher precision, dramatically reducing GPU memory during fine-tuning. Since the team already uses LoRA and wants to further reduce memory footprint, QLoRA is the correct combination. It is specifically designed as a memory-reduction technique layered on top of LoRA.

Answer

Data parallelism

Answer

Gradient checkpointing

Answer

Mixed precision