Techniques to Improve Generative AI Model Output →mediumMultiple Choice

Generative AI Leader Temperature Practice Question

Exhibit

Refer to the exhibit. The following is a configuration block from a Vertex AI PaLM API request:
{
  "instances": [{"context": "You are a helpful assistant.", "messages": [{"author": "user", "content": "Explain quantum computing"}]}],
  "parameters": {
    "temperature": 0.9,
    "maxOutputTokens": 1000,
    "topK": 40,
    "topP": 0.95,
    "candidateCount": 1
  }
}

A user reports that the model's response to the same prompt varies significantly across different calls. Which parameter change would most likely reduce variability?

⚠ Common exam trap

Candidates often mistake topK or candidateCount for the primary control of output variability, but in Google's Vertex AI and Gen AI models, temperature is the direct parameter that governs randomness in token selection.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Decrease temperature to 0.2.

Temperature controls the randomness of token sampling. Lowering temperature (e.g., to 0.2) makes the model's output more deterministic by reducing the probability of low-likelihood tokens, thus decreasing variability across calls for the same prompt.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Decrease topK to 10.
Why it's wrong here
Lowering topK restricts sampling to the ten highest-probability tokens, which narrows the distribution and reduces run-to-run variation. It is tempting because topK does control randomness, but temperature governs the sharpness of the probability distribution directly, making it the primary lever for this symptom.
✓
Decrease temperature to 0.2.
Why this is correct
Lowering temperature to 0.2 sharpens the softmax probability distribution, so high-probability tokens dominate sampling and near-deterministic output replaces the wide variance seen at higher values. This directly satisfies the stem's requirement to reduce run-to-run variability for an identical prompt, since temperature is the parameter governing sampling randomness.
✗
Increase candidateCount to 3.
Why it's wrong here
candidateCount returns multiple alternative responses per call; it increases the number of outputs rather than narrowing sampling, so variability across calls persists. It suits generating diverse options for ranking or selection. Reducing variability requires tightening the sampling distribution, for example lowering temperature.
✗
Increase maxOutputTokens to 2000.
Why it's wrong here
maxOutputTokens caps response length only; it does not affect how tokens are sampled, so run-to-run variability is unchanged. Raising it suits generating longer answers, such as detailed reports. Reducing variability requires adjusting sampling parameters, typically lowering temperature.

Go deeper

Generative AI Leader Temperature Practice Question

Exhibit

Why each option matters

Option-by-option breakdown

Related to this question