AI-900 Practice Question: Describe features of generative AI workloads on Azure

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✗

  temperature

  Why it's wrong here

  Temperature is a sampling parameter that scales the logits before the probability distribution is computed, controlling how random or deterministic the generated code is. A low value like 0.1 makes output focused and reproducible, while a high value like 0.9 increases diversity at the cost of reliability. It does not impose any bound on output length—it only alters which token sequences are likely—so it cannot cap the number of tokens returned.

• ✓

  max_tokens

  Why this is correct

  max_tokens sets a hard upper limit on the number of tokens the Azure OpenAI service will emit in a single completion, and generation stops as soon as that count is reached or the model produces an end-of-sequence marker. For code generation, this means the entire function, class, or script must fit within the allocated budget or the output will be truncated mid-statement. Increasing max_tokens is the only direct way to allow longer generated code; all other parameters affect content probability rather than total length.

• ✗

  top_p

  Why it's wrong here

  top_p, also called nucleus sampling, restricts the model to sample from the smallest set of tokens whose combined cumulative probability reaches the specified p value, such as 0.9, while discarding the long tail of unlikely tokens. This changes the diversity of token choices and can make output more focused or more varied, but it does not define any maximum on the number of tokens generated. A token limit like max_tokens is still required to stop generation; top_p alone cannot cap response length.

• ✗

  frequency_penalty

  Why it's wrong here

  frequency_penalty adjusts the logit scores of tokens by subtracting a penalty proportional to how often each token has already appeared in the current completion, thereby discouraging repetition of the same words, phrases, or code constructs. This is useful for reducing redundant lines of code, repetitive comments, or boilerplate, but it has no upper-bound effect on total output size. It may make the model write more concise code by avoiding repeats, yet it cannot directly control how many tokens are returned.