Generative AI Leader Practice Question: The primary purpose of the temperature parameter…

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✗

  It sets the maximum number of tokens the model can generate in a single response

  Why it's wrong here

  Maximum output length is set by the max tokens parameter, which caps generation independently of sampling. Temperature scales logits before softmax, altering randomness rather than length. The option tempts because both are generation settings, yet only max tokens truncates the response.

• ✗

  It determines the number of candidate tokens considered at each step

  Why it's wrong here

  Candidate token count is governed by top-k or nucleus sampling parameters, not temperature. Temperature rescales the probability distribution to control randomness. Confusion arises because both shape sampling, but top-k limits the pool while temperature flattens or sharpens it, so this describes a different control.

• ✗

  It adjusts the similarity threshold for vector search retrieval

  Why it's wrong here

  Vector search similarity thresholds belong to retrieval configuration, such as top-k or score cut-offs in a vector store, and operate before the model generates text. Temperature instead modifies the softmax distribution during decoding. The link is tempting because retrieval and generation both influence output, but they act at separate stages.

• ✓

  It controls the randomness of token selection; lower values make output more deterministic

  Why this is correct

  Temperature scales the probability distribution over the vocabulary before each token is sampled, so lower values sharpen that distribution towards the highest-probability tokens, yielding repeatable, deterministic output. This directly satisfies the stem's focus on the parameter's primary purpose: governing randomness in token selection.

Go deeper

Related to this question