NCP-GENL Fine-Tuning Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✗

  Increase the micro batch size to improve GPU utilization and reduce the number of optimizer steps.

  Why it's wrong here

  Increasing micro batch size raises activation memory per GPU, which is the opposite of reducing memory usage. While it can improve utilization, it often triggers out-of-memory errors on large models. The goal here is memory reduction, so larger micro batches are counterproductive unless paired with other memory-saving techniques.

• ✓

  Enable LoRA by setting adapter dimensions and target modules so only adapter weights receive gradients.

  Why this is correct

  LoRA freezes the base model and trains low-rank adapter matrices, which drastically reduces the number of trainable parameters and optimizer state. This lowers memory usage during fine-tuning while preserving the pretrained weights. In NeMo, this is configured through the PEFT section with adapter dimensions and target modules, making it a direct memory-saving measure.

• ✗

  Disable tensor parallelism so each GPU holds a full copy of the model weights.

  Why it's wrong here

  Disabling tensor parallelism would require each GPU to store the full model, which is impossible for a 70B parameter model on a single H100. Tensor parallelism is precisely what distributes model weights across GPUs. Removing it would increase per-GPU memory and likely cause immediate out-of-memory failures.

• ✗

  Switch from bfloat16 to float32 precision to improve numerical stability during training.

  Why it's wrong here

  Using float32 doubles the memory footprint of weights, activations, and optimizer states compared to bfloat16. While it can improve numerical stability, it directly contradicts the objective of reducing GPU memory usage. On H100 GPUs, bfloat16 is typically preferred for large model fine-tuning to save memory and increase throughput.

• ✓

  Enable gradient checkpointing to recompute activations during the backward pass instead of storing them.

  Why this is correct

  Gradient checkpointing trades compute for memory by storing only a subset of activations and recomputing the rest during backpropagation. This significantly reduces activation memory, which is often the largest consumer during fine-tuning of large models. It is a standard NeMo option and complements LoRA by further lowering peak memory.