NCA-GENL Experimentation Practice Question
During an ablation study on a retrieval-augmented LLM pipeline, the team removes the reranking stage and observes a large drop in answer accuracy on their benchmark. Before concluding that reranking is essential, which additional experiment is most important to run?
⚠ Common exam trap
The trap here is jumping from an observed accuracy drop to a causal claim about reranking without checking whether the comparison is confounded by other pipeline differences.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Run a control condition that keeps the pipeline identical except for a neutral change unrelated to reranking.
An ablation shows that a change in the pipeline correlates with an accuracy drop, but correlation is not causation unless other differences are ruled out. A control condition with a neutral modification tests whether the pipeline is sensitive to arbitrary changes. If the control shows little effect while removing reranking causes a large drop, the causal claim about reranking is substantially strengthened.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Run a control condition that keeps the pipeline identical except for a neutral change unrelated to reranking.
Why this is correct
A control isolates the effect of the intervention from other differences between the ablated and baseline runs. If a neutral modification produces little change while removing reranking produces a large drop, the conclusion that reranking drives accuracy is much stronger. This is the key validity check before attributing the effect to the removed component.
- ✗
Increase the number of retrieved documents to compensate for the missing reranker.
Why it's wrong here
Changing the retrieval depth alters another part of the pipeline, so any accuracy change would be due to that modification rather than confirming the reranker's role. It also does not validate the original ablation. Compensation experiments are useful for engineering but do not substitute for a control that isolates the ablated component.
- ✗
Replace the benchmark with a larger one to increase statistical power.
Why it's wrong here
A larger benchmark reduces measurement variance but does not rule out confounding between the baseline and ablated configurations. The central question is whether removing reranking, and not some other difference, caused the drop. Scaling the evaluation set is a secondary improvement, not the critical validity check the team needs first.
- ✗
Re-run the ablation with a different random seed to see if the accuracy drop persists.
Why it's wrong here
Seed variation checks run-to-run noise but does not address whether the drop is specifically caused by removing reranking. If the benchmark or retrieval corpus differs in other ways, the result remains confounded. Seed repetition is useful but secondary to verifying that the comparison isolates the reranking component itself.
About these practice questions
Courseiva writes every NCA-GENL question from scratch — 367 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official NVIDIA exam blueprint
This NCA-GENL practice question is part of Courseiva's free NVIDIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the NCA-GENL exam.