AI-900 Practice Question: Describe features of generative AI workloads on Azure

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✗

  The operating temperature of the GPU hardware running the model

  Why it's wrong here

  In AI terminology, temperature is a model hyperparameter that scales the logits (raw scores) before the softmax layer to shape token probabilities; it has nothing to do with the physical temperature of GPU silicon. GPU operating temperature is a hardware cooling concern that affects clock speeds, thermal throttling, and system reliability, not the model's output distribution. Even if the GPU runs hot, the random seed and sampling logic will produce the same probabilities — though excessive heat might degrade performance or shorten hardware life.

• ✓

  A parameter controlling the randomness and creativity of model outputs

  Why this is correct

  Temperature is the parameter that controls the randomness and creativity of a language model's output by scaling the logits before applying the softmax function. With a low value (e.g., 0.1), the probability distribution becomes more peaked, so the model almost always picks the most likely token, producing deterministic, repetitive text. With a high value (e.g., 1.5), the distribution is flattened, encouraging varied, surprising, and sometimes less coherent outputs. As temperature approaches zero from above, sampling becomes equivalent to greedy decoding, while exactly zero typically forces a division-by-zero and is avoided in practice.

• ✗

  The time required to generate a response

  Why it's wrong here

  Response generation time, or inference latency, is a performance metric driven by model architecture, sequence length, batch size, and hardware capability — not by the temperature setting. Temperature changes the sampling behavior of the token-generation loop: at high temperatures the model picks less probable tokens more often, but the arithmetic cost of scaling logits and drawing a sample is negligible and constant. Thus, a model set to temperature 0.2 versus 2.0 will take essentially the same wall-clock time to produce a token (assuming both complete), because temperature alters probability weights, not the speed of computation.

• ✗

  The minimum confidence threshold for a response

  Why it's wrong here

  A minimum confidence threshold is a decision rule used in classification or retrieval to accept a prediction only if its probability is above a set cut-off, often for quality control or safety. In contrast, temperature controls the shape of the probability distribution over the entire vocabulary by dividing the logits — higher temperature increases entropy and makes diverse tokens more likely, while lower temperature concentrates probability on the most likely token. Unlike a threshold, temperature doesn't filter out low-confidence responses; it reweights the odds of each token being sampled, and a low-temperature model can still produce an answer with low absolute probability.

Go deeper

Related to this question