NCP-GENL LLM Architecture Practice Question
Which TWO of the following statements correctly describe the role of Grouped Query Attention (GQA) in modern LLM architectures?
⚠ Common exam trap
Candidates often mistake GQA for a training-only optimization or a method that increases accuracy. GQA is specifically designed for inference-time efficiency by reducing memory bandwidth requirements.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
GQA reduces the memory footprint of the KV cache during inference.
GQA balances the performance of Multi-Head Attention (MHA) and the memory efficiency of Multi-Query Attention (MQA). By sharing keys and values across groups of heads, it reduces the size of the KV cache during inference. This is crucial for high-throughput deployment environments, as it optimizes memory bandwidth utilization without sacrificing the granular representational capacity that individual query heads provide.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
GQA eliminates the need for separate query, key, and value projection layers.
Why it's wrong here
GQA still utilizes distinct projections for queries, keys, and values. It simply reduces the number of unique key and value heads relative to the number of query heads. The projection architecture remains intact, ensuring that the model retains its ability to attend to complex patterns across different subspaces.
- ✓
GQA reduces the memory footprint of the KV cache during inference.
Why this is correct
By reducing the total number of key and value heads, the size of the KV cache stored in GPU memory is significantly decreased. This reduction is highly beneficial for serving models at scale, as it allows for larger batch sizes or longer context lengths without exceeding available memory capacity.
- ✗
GQA improves model training speed by increasing the number of compute operations.
Why it's wrong here
GQA is designed for efficiency rather than increased compute; it actually reduces the number of operations required for key and value updates. The primary benefit is speed during inference due to memory bandwidth constraints, rather than an increase in computational volume during the training phase of the model.
- ✓
GQA achieves a compromise between Multi-Head Attention and Multi-Query Attention.
Why this is correct
MHA is memory-intensive, while MQA is memory-efficient but can suffer from quality degradation. GQA groups heads to provide better performance than pure MQA while remaining significantly more memory-efficient than MHA. This makes it a standard choice for modern LLMs that need to be both performant and efficient.
- ✗
GQA prevents the model from using positional embeddings.
Why it's wrong here
GQA is entirely agnostic to the type of positional embedding used. It works perfectly alongside RoPE, ALiBi, or absolute positional embeddings. The grouping mechanism is applied at the attention head level, while positional information is injected into the queries and keys independently of how those heads are grouped.
About these practice questions
One of 352 original NCP-GENL practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official NVIDIA exam blueprint
This NCP-GENL practice question is part of Courseiva's free NVIDIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the NCP-GENL exam.