NCA-GENL Experimentation Practice Question
An engineer is evaluating a RAG-based assistant and wants to isolate whether retrieval quality or the generator is responsible for wrong answers. They build a small labeled set of questions with known correct passages and known correct answers. Which experimental design most cleanly separates the contribution of the retriever from that of the generator?
⚠ Common exam trap
The trap here is using end-to-end accuracy as if it were a diagnostic, when it aggregates two independent failure sources into one number.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Measure retriever recall against the known correct passages, then feed the known correct passages to the generator and measure answer accuracy separately.
Isolating retriever versus generator contributions requires scoring each stage against ground truth. Retriever recall measured against known correct passages reveals retrieval quality, and feeding those oracle passages to the generator reveals generation quality independent of retrieval. End-to-end comparisons and top-k tuning mix the two stages and cannot attribute failures.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Measure retriever recall against the known correct passages, then feed the known correct passages to the generator and measure answer accuracy separately.
Why this is correct
By scoring retrieval against ground-truth passages and separately scoring generation given oracle passages, the design isolates each component's contribution. Retrieval recall shows whether the right context is found, and oracle-context accuracy shows whether the generator can use correct context. This controlled decomposition is the standard way to attribute failures in RAG systems.
- ✗
Compare the end-to-end accuracy of the RAG system against the same generator without retrieval.
Why it's wrong here
This comparison shows whether retrieval helps overall but does not separate retrieval quality from generation quality when both are imperfect. A RAG system can underperform a closed-book model because of poor retrieval or because the generator mishandles context, and this design cannot tell which. It answers a different question than the one posed.
- ✗
Increase the retriever top-k until end-to-end accuracy stops improving, then freeze the retriever and tune the generator.
Why it's wrong here
Tuning top-k improves recall but conflates the two stages because the generator still consumes retrieved context, which may include distractors. The resulting accuracy change cannot be attributed to retrieval alone. This is an optimization loop, not a controlled experiment that isolates component contributions.
- ✗
Run the full pipeline and measure end-to-end answer accuracy, then retrain the generator on the failures.
Why it's wrong here
End-to-end accuracy conflates retrieval misses with generation errors, so it cannot attribute failures to either component. Retraining on failures without knowing the cause risks fixing the wrong subsystem and can degrade performance elsewhere. This design produces a number but not the diagnostic separation the engineer needs.
About these practice questions
Courseiva writes every NCA-GENL question from scratch — 367 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official NVIDIA exam blueprint
This NCA-GENL practice question is part of Courseiva's free NVIDIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the NCA-GENL exam.