NCA-GENL Experimentation Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✗

  Measure end-to-end inference latency with NVIDIA Triton Inference Server metrics.

  Why it's wrong here

  Latency metrics describe serving performance, not the diversity of generated text. Changing top-p has minimal effect on latency because it alters token selection, not the compute path. Using latency as the evaluation signal would tell the engineer nothing about whether outputs became more varied, so it does not support the stated experiment.

• ✓

  Compute distinct-n and self-BLEU across a fixed prompt set for each top-p value.

  Why this is correct

  Distinct-n measures lexical variety and self-BLEU measures similarity among generated outputs, both computed automatically over a fixed prompt set. Together they quantify diversity changes as top-p varies, giving the engineer an objective, repeatable signal. This matches the goal of measuring output diversity without depending solely on subjective human ratings.

• ✗

  Compare training loss curves from the original fine-tuning job.

  Why it's wrong here

  Training loss curves describe how the model fit its training data during fine-tuning. They are fixed once training ends and are unaffected by inference-time sampling parameters like top-p. Reviewing them cannot show how output diversity shifts across settings, so this approach is unrelated to the inference experiment.

• ✗

  Track GPU utilization and memory bandwidth during generation with Nsight Systems.

  Why it's wrong here

  GPU utilization and memory bandwidth are hardware efficiency indicators that do not reflect the semantic or lexical diversity of model outputs. Top-p changes which tokens are sampled, leaving resource usage largely unchanged. Profiling would not reveal whether the generated support answers became more varied, so it cannot answer the experiment's question.