NCP-GENL LLM Architecture Practice Question
Exhibit
Error Log: [CUDA_ERROR_OUT_OF_MEMORY] during attention calculation. Sequence length: 128k. Model: 70B parameter, FP16.
Refer to the exhibit. The model is encountering an OOM error during long-context processing. Which architectural adjustment is most appropriate to resolve this while maintaining context length?
⚠ Common exam trap
Candidates often suggest reducing the batch size or model precision. While these help, they do not address the fundamental quadratic memory growth of attention that FlashAttention-2 is designed to solve.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Switch to FlashAttention-2 kernels to optimize memory usage.
When sequence lengths scale to 128k, the attention matrix size grows quadratically, consuming massive memory. Implementing FlashAttention-2 or similar memory-efficient attention kernels is the industry-standard solution. These kernels optimize the memory layout and tile operations to compute attention without materializing the massive N×N matrix, drastically reducing peak memory usage and enabling the processing of very long sequences within existing GPU capacity.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Increase the number of hidden layers.
Why it's wrong here
Adding more hidden layers increases the model's overall memory requirement, as each layer contains additional weight matrices and cache space. This would worsen the OOM error rather than resolve it. Layer depth is generally fixed after training, and increasing it would not help with memory-bound sequence length constraints.
- ✓
Switch to FlashAttention-2 kernels to optimize memory usage.
Why this is correct
FlashAttention-2 provides a fused kernel that computes attention in blocks, avoiding the storage of the full attention matrix in VRAM. This is the optimal way to handle long sequences like 128k, as it solves the memory bottleneck at the architectural kernel level without sacrificing model capability.
- ✗
Reduce the embedding dimension size.
Why it's wrong here
The embedding dimension is a fixed hyperparameter determined during the model's initial training. Modifying it post-training would require re-training the entire model, as the weight matrices would no longer align. It is not a viable strategy for resolving inference-time memory issues on an already trained model.
- ✗
Disable the KV cache entirely.
Why it's wrong here
Disabling the KV cache would make inference extremely slow, as the model would have to recompute keys and values for every new token. While it saves memory, it is not a practical solution for generative tasks. Furthermore, it does not solve the underlying issue of the massive attention computation.
Visual reference
About these practice questions
This NCP-GENL question is part of Courseiva's 352-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official NVIDIA exam blueprint
This NCP-GENL practice question is part of Courseiva's free NVIDIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the NCP-GENL exam.