A quality assurance team at a software company uses Azure OpenAI Service to generate compliance reports. They need the model to produce the exact same output for a given prompt every time the API is called, to ensure reproducibility during testing. Which parameter should they set to achieve this deterministic behavior?
Trap 1: Set frequency penalty to 1
Frequency penalty reduces repetition of tokens but does not eliminate randomness; outputs may still vary.
Trap 2: Set top_p to 1
Top_p controls nucleus sampling; setting it to 1 includes all tokens, which still allows random choices and does not guarantee determinism.
Trap 3: Set max_tokens to the expected output length
Max_tokens limits the length of the output but does not affect the randomness of token selection.
- A
Set temperature to 0
Temperature controls randomness; setting it to 0 makes the model choose the most likely token every time, producing deterministic outputs.
- B
Set frequency penalty to 1
Why wrong: Frequency penalty reduces repetition of tokens but does not eliminate randomness; outputs may still vary.
- C
Set top_p to 1
Why wrong: Top_p controls nucleus sampling; setting it to 1 includes all tokens, which still allows random choices and does not guarantee determinism.
- D
Set max_tokens to the expected output length
Why wrong: Max_tokens limits the length of the output but does not affect the randomness of token selection.