AI-900 Practice Question: Describe features of generative AI workloads on Azure
What is 'top_p' (nucleus sampling) in Azure OpenAI and how does it differ from temperature?
⚠ Common exam trap
Candidates often confuse top_p with a confidence or correctness threshold, when in fact it is a sampling parameter that controls the diversity of token selection by truncating the probability distribution.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Restricting token selection to those whose cumulative probability reaches p — an alternative diversity control to temperature
Top_p (nucleus sampling) in Azure OpenAI controls diversity by selecting tokens from the smallest set whose cumulative probability exceeds the threshold p, rather than sampling from the full probability distribution. This differs from temperature, which scales the logits before the softmax to flatten or sharpen the distribution; top_p dynamically cuts off the long tail of low-probability tokens, providing an alternative method to control randomness without affecting the relative ranking of high-probability tokens.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
The maximum percentage of the context window used for generating output
Why it's wrong here
Context window allocation governs the maximum number of tokens the model can process in prompt plus generated output, managed separately from sampling. top_p does not reserve or cap any percentage of that context; it operates only on the probability distribution for the next token at each generation step. The output length limit is typically a separate parameter like max_tokens, not top_p.
- ✓
Restricting token selection to those whose cumulative probability reaches p — an alternative diversity control to temperature
Why this is correct
Nucleus sampling with top_p works by sorting all next-token probabilities in descending order and keeping the smallest set whose cumulative probability is at least p, then renormalizing and sampling from that set. For example, top_p=0.9 means the model considers only tokens that together hold 90% of the probability mass, dynamically expanding or shrinking the candidate pool. This is an alternative to temperature for controlling diversity, because it adapts to the shape of the distribution rather than scaling logits uniformly.
- ✗
The probability threshold above which the model considers a response correct
Why it's wrong here
This treats top_p as an evaluation threshold for response correctness, which is a misinterpretation. Correctness is assessed by separate metrics like accuracy, BLEU, or human judgment, outside the generation process. top_p does not cause the model to deem a response correct or incorrect; it only influences the randomness and diversity of token selection during sampling.
- ✗
A parameter setting the minimum confidence before the model outputs a response
Why it's wrong here
This confuses top_p with a confidence threshold, which is a classification concept where a model rejects low-confidence predictions. top_p is a generative sampling parameter that selects a subset of tokens based on their cumulative probability mass, not a gate for whether the model responds at all. Even if the model is uncertain, top_p still produces output, just with a dynamically sized token pool.
Go deeper
Related to this question
About these practice questions
One of 985 original AI-900 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-900 exam.