AI0-001 AI Infrastructure and Technologies Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✗

  Converting the model to ONNX format before training

  Why it's wrong here

  ONNX is an interchange format aimed at portable inference across runtimes and hardware. It does not provide a training-time memory optimization, and exporting a model to ONNX is a post-training step. Using it here would not reduce the optimizer and activation memory consumed during fine-tuning, so it cannot address the out-of-memory failure the team is facing.

• ✗

  Quantization-aware training of the base weights at 4-bit

  Why it's wrong here

  Quantization-aware training simulates low-precision arithmetic so a model can later run efficiently in inference, and it is typically applied to the forward path. It does not by itself eliminate the optimizer state and gradient memory that full fine-tuning of a 7B model requires, so it would not resolve the out-of-memory condition described during training on this hardware.

• ✓

  Gradient checkpointing

  Why this is correct

  Gradient checkpointing discards most intermediate activations during the forward pass and recomputes them during backpropagation. This trades additional compute for a large reduction in activation memory, which is a major consumer at long sequence lengths. Combined with parameter-efficient tuning, it helps fit the fine-tuning job into the four A100 40 GB GPUs available to this team.

• ✓

  LoRA (Low-Rank Adaptation)

  Why this is correct

  LoRA freezes the pretrained weights and injects small trainable low-rank matrices into selected layers, so gradients and optimizer states are only maintained for those adapters. That removes the largest memory consumers of full fine-tuning, the optimizer moments for billions of parameters, and lets a 7B model train within the memory of four 40 GB GPUs while preserving base model quality.

• ✗

  Increasing the global batch size

  Why it's wrong here

  Raising the global batch size increases the number of activations and gradients held at once, which raises memory pressure rather than lowering it. The team is already exceeding GPU memory, so a larger batch would make the out-of-memory condition worse. Batch size is a throughput and convergence knob here, not a memory-reduction technique.