NCA-GENL Experimentation Practice Question
A research team is running a hyperparameter sweep for LoRA fine-tuning of a 13B-parameter LLM on NVIDIA GPUs. They notice that runs with identical configurations sometimes produce noticeably different evaluation scores, and the variance is larger than the differences between some of the hyperparameter settings being compared. Which action best addresses this problem?
⚠ Common exam trap
The trap here is believing that more runs or a best-of-N summary solves variance, when only replication with controlled seeds and uncertainty reporting makes the comparison valid.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Run each configuration with multiple fixed random seeds and compare mean evaluation scores with their confidence intervals.
When run-to-run variance exceeds the differences between configurations, single-run comparisons cannot support a conclusion. Repeating each configuration with several fixed seeds and reporting mean scores with confidence intervals turns noise into a measurable uncertainty, so the team can tell whether a hyperparameter effect is real. This is the standard remedy for high-variance sweeps and prevents selecting configurations on the basis of lucky seeds.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Report only the single best score achieved by each configuration, since the best result shows the model's potential.
Why it's wrong here
Reporting the maximum of noisy runs is a biased estimator that inflates apparent performance and hides instability. Configurations that happened to get a lucky seed would look superior, so the sweep would select for randomness rather than for genuinely better hyperparameters, worsening the very problem the team observed.
- ✗
Increase the learning rate for all runs so the model converges faster and the noise is averaged out.
Why it's wrong here
Raising the learning rate changes the optimization trajectory and can increase instability rather than reduce run-to-run variance. It also alters the very hyperparameter being studied, so the sweep no longer isolates the intended variables. Faster convergence does not fix the underlying seed and nondeterminism issues causing the spread.
- ✓
Run each configuration with multiple fixed random seeds and compare mean evaluation scores with their confidence intervals.
Why this is correct
Repeating each configuration across several fixed seeds converts a single noisy observation into a distribution, and comparing means with confidence intervals reveals whether a hyperparameter effect exceeds run-to-run noise. This directly addresses the reported problem, where variance is larger than the differences between settings, by quantifying uncertainty before drawing conclusions.
- ✗
Reduce the number of sweep configurations so fewer comparisons are affected by variance.
Why it's wrong here
Cutting the sweep shrinks the search space but leaves the variance untouched, so the remaining comparisons are just as unreliable. The team would trade away coverage of the hyperparameter space without gaining confidence in any individual result, which defeats the purpose of the sweep.
About these practice questions
One of 367 original NCA-GENL practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official NVIDIA exam blueprint
This NCA-GENL practice question is part of Courseiva's free NVIDIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the NCA-GENL exam.