NCA-GENL Experimentation Practice Question
A team is evaluating an LLM for a customer-support summarization task. They want to compare three prompt templates. Which experimental design most directly isolates the effect of the prompt template?
⚠ Common exam trap
The trap here is believing that testing each prompt with its own tuned decoding settings gives a fairer comparison.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use the same model, decoding parameters, and evaluation dataset for all three templates, changing only the template text.
A controlled comparison requires that only the variable of interest changes between conditions. By fixing the model, decoding parameters, and evaluation data and varying only the prompt template text, the team ensures that observed differences in summary quality are attributable to the template. This makes the experiment reproducible and the conclusions defensible.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Use a different model for each template to see which combination performs best overall.
Why it's wrong here
Changing the model along with the template confounds two variables. If one combination wins, you cannot tell whether the template or the model caused it. This defeats the goal of isolating the prompt template effect and makes the comparison uninterpretable.
- ✓
Use the same model, decoding parameters, and evaluation dataset for all three templates, changing only the template text.
Why this is correct
Holding model, decoding parameters, and dataset constant while varying only the template text isolates the template as the independent variable. Any measured difference can then be attributed to the template rather than to confounds. This is the core principle of a controlled experiment and directly answers the team's question.
- ✗
Vary temperature and top-p across templates so each template is tested under its own best decoding settings.
Why it's wrong here
Allowing decoding parameters to change per template introduces another source of variation. A template might appear better only because it received a more favorable temperature. The design no longer isolates the template, so causal claims about prompt wording become impossible.
- ✗
Evaluate each template on a different dataset to cover more customer scenarios.
Why it's wrong here
Different datasets differ in difficulty and domain, so score differences could reflect data rather than template quality. A fair comparison requires the same inputs across conditions. Using separate datasets increases coverage but destroys the controlled comparison the team needs.
About these practice questions
One of 367 original NCA-GENL practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official NVIDIA exam blueprint
This NCA-GENL practice question is part of Courseiva's free NVIDIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the NCA-GENL exam.