NCA-GENL Core Machine Learning and AI Knowledge Practice Question
An enterprise machine learning team is training a large-scale transformer model on a cluster of NVIDIA A100 GPUs using mixed precision (FP16). During the initial training phase, the team notices sudden numerical underflow resulting in vanishing gradients and stalled loss convergence. Which optimization technique must be applied to mitigate this issue without sacrificing the memory-efficiency benefits of FP16?
⚠ Common exam trap
Candidates frequently confuse loss scaling with learning rate warm-up or gradient clipping. While learning rate warm-up stabilizes early training dynamics and gradient clipping prevents exploding gradients, neither addresses the precision representational underflow caused by the limited dynamic range of FP16.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Applying dynamic loss scaling to the loss value prior to backpropagation to shift gradients into the representational range.
Loss scaling multiplies the forward pass loss by a scaling factor to shift small gradient magnitudes into the representational range of the FP16 format, preventing underflow. This is essential in NVIDIA mixed-precision training because FP16 has a narrow dynamic range compared to FP32. Proper scaling prevents gradient values from truncating to zero while retaining half-precision throughput and memory footprint advantages on Tensor Cores.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Implementing gradient clipping by global norm to bound large gradient updates before parameter application.
Why it's wrong here
Gradient clipping bounds excessively large gradient values to prevent exploding gradients during optimization steps. However, it does not prevent numerical underflow or vanishing gradients caused by the restricted dynamic range of the half-precision FP16 data format during backpropagation computations.
- ✗
Switching the entire training cluster to FP32 precision to guarantee maximum numerical stability and range.
Why it's wrong here
FP32 doubles memory consumption and halves throughput, discarding the memory-efficiency FP16 provides, so it fails the stated constraint. It is tempting as a guaranteed-stability fallback, but loss scaling is the correct technique: it preserves FP16 storage while scaling gradients to prevent underflow.
- ✓
Applying dynamic loss scaling to the loss value prior to backpropagation to shift gradients into the representational range.
Why this is correct
Dynamic loss scaling automatically adjusts a multiplier on the loss value to maintain gradient magnitudes within the representational limits of FP16. This prevents underflow during backward passes without requiring full FP32 precision across all network layers.
- ✗
Increasing the mini-batch size exponentially to smooth out noisy gradient estimates across distributed nodes.
Why it's wrong here
Larger batches reduce gradient noise but do nothing for FP16's limited exponent range, where small gradient values flush to zero. The remedy is loss scaling, which multiplies the loss before backpropagation and unscales gradients afterwards. Batch sizing addresses variance and throughput, not representational underflow.
About these practice questions
One of 367 original NCA-GENL practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official NVIDIA exam blueprint
This NCA-GENL practice question is part of Courseiva's free NVIDIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the NCA-GENL exam.