NCP-GENL Fine-Tuning Practice Question
An ML engineer is fine-tuning a 13B LLM with LoRA on 4 NVIDIA A100 GPUs using NVIDIA NeMo. They notice that the effective batch size is very small and gradients are noisy, but increasing the per-GPU micro batch size triggers out-of-memory errors. Which technique should they apply to increase the effective batch size without increasing memory per step?
⚠ Common exam trap
The trap here is equating adapter capacity or numeric precision with batch size, when only gradient accumulation increases effective batch size without raising peak memory.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Enable gradient accumulation to sum gradients across multiple micro batches before the optimizer step.
Gradient accumulation performs multiple forward and backward passes on small micro batches and sums their gradients before one optimizer update, emulating a larger batch while keeping peak memory at the level of a single micro batch. It is the standard remedy when memory prevents enlarging the micro batch. Changing adapter rank, precision, or sequence length does not increase the number of samples per update in a memory-neutral way.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Switch from bfloat16 to float32 precision to stabilize gradient values.
Why it's wrong here
Using float32 doubles activation and parameter memory relative to bfloat16, which would make out-of-memory errors more likely, not less. While higher precision can reduce numerical noise, the scenario attributes noisy gradients to small batch size. This change does not increase the number of samples per update and worsens the memory constraint that already blocks larger micro batches.
- ✓
Enable gradient accumulation to sum gradients across multiple micro batches before the optimizer step.
Why this is correct
Gradient accumulation executes several forward and backward passes on small micro batches, accumulating gradients before applying a single optimizer update. This raises the effective batch size without increasing peak memory, because only one micro batch resides in memory at a time. It directly addresses the noisy-gradient problem while respecting the memory ceiling that prevents larger micro batches.
- ✗
Increase the LoRA rank and alpha so the adapter captures more information per step.
Why it's wrong here
LoRA rank and alpha control adapter capacity and scaling, not the number of samples contributing to each optimizer update. Raising them adds trainable parameters and can increase memory slightly, but it does not enlarge the effective batch size or reduce gradient noise from small batches. This change addresses model expressiveness rather than optimization stability.
- ✗
Reduce the sequence length by truncating all training examples to 128 tokens.
Why it's wrong here
Truncating sequences lowers activation memory and could allow a larger micro batch, but it discards information and may cut off target responses, harming quality. It is a data-altering workaround rather than a mechanism to increase effective batch size. The scenario asks for a technique that raises effective batch size without changing data, which truncation does not satisfy.
About these practice questions
Courseiva writes every NCP-GENL question from scratch — 352 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official NVIDIA exam blueprint
This NCP-GENL practice question is part of Courseiva's free NVIDIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the NCP-GENL exam.