NCA-GENL Experimentation Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✗

  Increase the batch size to the maximum the GPU memory allows so the baseline finishes quickly.

  Why it's wrong here

  A larger batch size speeds the run but changes the optimization dynamics and effective learning rate, so the baseline no longer represents the intended configuration. Speed alone does not create a repeatable reference point, and memory-driven batch sizing makes the run hardware-dependent rather than a controlled starting condition.

• ✓

  Fix a random seed in the NeMo training configuration and record the exact dataset version, model checkpoint, and NeMo container tag used.

  Why this is correct

  Pinning the seed plus capturing dataset version, starting checkpoint, and container tag makes the baseline reproducible; re-running with the same inputs yields a comparable result. Without this record, later sweeps cannot be attributed to the hyperparameter change, because data or environment drift would confound the comparison.

• ✗

  Enable automatic mixed precision and Tensor Core acceleration to reduce training time.

  Why it's wrong here

  Mixed precision and Tensor Core use are performance optimizations, not reproducibility controls. They can subtly shift numerical results between runs or hardware generations, so enabling them without recording the setting undermines a fair baseline. The scenario asks for a repeatable reference, which requires capturing configuration, not just accelerating the job.

• ✗

  Run the experiment three times and report the best validation loss observed.

  Why it's wrong here

  Selecting the best of several runs introduces selection bias and hides variance, so the reported number is not a stable baseline and cannot be fairly compared with later sweeps. Reproducibility comes from controlling inputs and recording them, not from picking a favorable outcome after the fact.