AI-900 Practice Question: Describe features of generative AI workloads on Azure
A developer is using Azure OpenAI with GPT-4 to build a chatbot that answers legal questions based on a company's internal policy documents. The developer wants the model's responses to be maximally deterministic and factual, avoiding any creative or speculative language. Which parameter should the developer set to the lowest possible value in the API call?
⚠ Common exam trap
Many candidates confuse 'randomness' with 'repetition' or 'topic diversity,' leading them to choose frequency or presence penalties, but those parameters do not enforce deterministic factual output—only temperature set to 0 does.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Temperature
Temperature controls the randomness of the model's output. Setting it to the lowest possible value (0) makes the model deterministic, always choosing the most likely next token, which is ideal for factual, non-creative responses like legal answers. Higher temperature values introduce variability and creativity, which would be undesirable for this use case.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Temperature
Why this is correct
Temperature in Azure OpenAI GPT-4 directly scales the logits (the raw scores) before the softmax layer, effectively sharpening or flattening the probability distribution over the next token. Setting it closer to 0, such as 0.0 or 0.1, forces the model to almost always pick the highest-probability token, yielding highly deterministic and factual output. This is the most direct and intuitive control knob for reducing creative or speculative language in a chatbot that must answer with accurate, grounded responses.
- ✗
Frequency penalty
Why it's wrong here
Frequency penalty is a logit adjustment that subtracts a value proportional to how many times a token has already appeared in the generated sequence, which specifically suppresses repeated words or phrases. It operates on token frequency rather than the overall shape of the probability distribution, so it does not reduce the sampling randomness itself. Even with a high frequency penalty, the model can still choose widely different tokens across runs, so it is not a reliable way to make outputs deterministic.
- ✗
Presence penalty
Why it's wrong here
Presence penalty applies a fixed single penalty to any token that has appeared at least once, regardless of how often it occurred, thereby encouraging the model to introduce new topics and vocabulary. This alters which tokens are favored based on novelty, but it does nothing to change the underlying sampling temperature or the sharpness of the probability curve. Consequently, it can increase diversity and topical breadth but cannot guarantee that answers are more factual or reproducible.
- ✗
Top_p
Why it's wrong here
Top_p, also called nucleus sampling, works by dynamically truncating the candidate set to the smallest subset of tokens whose cumulative probability mass exceeds the threshold p, and then sampling only from that subset. While setting a low top_p can reduce output variety by limiting the pool of likely tokens, it remains a less direct and less granular control than temperature for producing deterministic answers. In Azure OpenAI, adjusting temperature is the standard recommendation for making responses more factual and reproducible, and combining top_p with temperature can lead to unexpected interactions.
Go deeper
Related to this question
Learn chapter
Azure Machine Learning Studio
Key term
Model
In IT and AI, a model is a trained mathematical representation that learns patterns from data to make predictions or decisions.
Key term
Token
A token is a digital or physical object that represents a set of permissions or access rights, used to verify identity or grant access in computer systems.
About these practice questions
Courseiva writes every AI-900 question from scratch — 985 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-900 exam.