Cost Reduction Strategies for Azure OpenAI Service
Which TWO actions can you take to reduce the cost of using Azure OpenAI Service for a chat application?
Quick Answer
Azure OpenAI Service billing is tied directly to token count, both the tokens sent in and the tokens generated back, so the two levers that reduce cost are the two things that determine how many tokens get billed: how large a model is doing the generating, and how long its responses are allowed to run. Switching from GPT-4 to a smaller model like GPT-3.5 for a chat application reduces per-token pricing significantly, since larger models cost more per token to reflect their greater capability — a tradeoff worth making when the smaller model's quality is sufficient for the use case. Separately, capping the max_tokens parameter puts a hard ceiling on how many tokens a single completion can generate, which prevents the model from producing unnecessarily long responses that consume tokens (and cost) without adding proportional value to a chat interaction. These two actions attack cost from different angles — one lowers the price per token, the other lowers the number of tokens generated — which is why they're commonly paired as the two-part answer rather than either alone being sufficient. Any scenario asking specifically about reducing Azure OpenAI cost, as opposed to improving output quality or accuracy, should be evaluated against model size and response length as the two primary cost levers.
⚠ Common exam trap
Test-takers frequently confuse cost-saving techniques with performance-tuning parameters, mistakenly thinking that adjusting penalty settings or content filtering reduces token consumption, when in fact only limiting token output or using a cheaper model directly lowers the bill.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Set the max_tokens parameter to a lower value.
Reducing the max_tokens parameter directly limits the number of tokens generated per API call, which lowers the cost since Azure OpenAI Service bills per token (both input and output). By capping the response length, you avoid paying for unnecessarily long completions.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Increase the frequency penalty.
Why it's wrong here
Frequency penalty does not reduce cost.
- ✗
Enable content filtering.
Why it's wrong here
Content filtering does not affect cost.
- ✓
Set the max_tokens parameter to a lower value.
Why this is correct
Reduces token count per response.
- ✗
Increase the max_tokens parameter to allow longer responses.
Why it's wrong here
Increases token usage.
- ✓
Use a smaller model like GPT-3.5 instead of GPT-4.
Why this is correct
Smaller models are cheaper.
About these practice questions
This AI-102 question is part of Courseiva's 761-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on AI-102
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. Which TWO actions should you take to reduce the cost of using Azure OpenAI for a chatbot that handles high traffic?
medium- A.Increase max_tokens to reduce the number of requests.
- ✓ B.Use the batch API for non-real-time requests.
- ✓ C.Implement caching for frequently asked questions.
- D.Lower the temperature to 0 to reduce token usage.
- E.Fine-tune the model to reduce prompt length.
Why B: Option B is correct because the Azure OpenAI Batch API processes asynchronous, non-real-time workloads at a 50% discount compared to standard global pricing, so routing any chatbot requests that don't need immediate responses through batch jobs directly lowers per-token cost. Option C is correct because caching responses to frequently asked questions (for example with Azure Cache for Redis or a semantic cache) means repeated identical or similar prompts are served without calling the model at all, eliminating token charges for that traffic and reducing overall request volume. Option A is not correct because increasing max_tokens raises the maximum output length and therefore can increase, not decrease, token consumption and cost. Option D is not correct because temperature controls randomness in sampling, not the number of tokens generated, so setting it to 0 does not reduce token usage or cost. Option E is not correct because fine-tuning does not shorten the prompt by itself and adds training and hosting costs; prompt length is reduced through techniques like prompt compression or shorter system messages, not fine-tuning alone.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-102 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-102 exam.