Describe features of Natural Language Processing workloads on Azure →hardMultiple ChoiceObjective-mapped
AI-900 Practice Question: Describe features of Natural Language Processing workloads on Azure
What is 'token budget' and 'context window' management in large language models?
⚠ Common exam trap
Test-takers frequently confuse operational or billing limits (subscription spending, rate limits, pre-purchased tokens) with the model's inherent architectural constraint, which is the context window and token budget for a single API call.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
The maximum text the model can process in one call, requiring strategic management of what fits in context
'token budget' and 'context window' refer to the maximum number of tokens (words, subwords, or characters) a large language model can process in a single inference call. The context window is a fixed limit (e.g., 4096 tokens for GPT-3.5, 8192 for GPT-4), and managing the token budget involves strategically truncating, summarizing, or prioritizing input text to fit within this window, ensuring the model can generate coherent and relevant responses without exceeding its capacity.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
The monthly spending limit set for an Azure OpenAI subscription
Why it's wrong here
This is an Azure billing control at the subscription or resource level, not a property of the model's context window. It caps the total dollar amount that can be consumed across all API calls in a billing period, whereas a token budget refers to the limited number of tokens (input + output) that can fit into a single request's context window. Confusing financial budgeting with the model's finite context capacity leads to a fundamental misunderstanding of how Azure OpenAI processes text.
- ✓
The maximum text the model can process in one call, requiring strategic management of what fits in context
Why this is correct
The token budget is the finite context window capacity (e.g., 128K tokens for GPT-4 Turbo) that limits the combined length of the system prompt, few-shot examples, conversation history, and user input in a single request. Developers must strategically allocate this space—deciding what to truncate, summarize, or exclude—so that the model has enough room for a coherent and accurate completion. This is a core prompt-engineering concept for managing context overflow and ensuring relevant information is present.
- ✗
The number of API calls allowed per minute before rate limiting kicks in
Why it's wrong here
That is a rate limit, typically expressed in requests per minute (RPM) or tokens per minute (TPM) for throttling and quota management. A token budget is not about how many calls you can make in a second, but about how many tokens you can include in a single API call's prompt and completion, constrained by the model's context window. Rate limits are enforced by the Azure infrastructure, while the token budget is a per-request model limitation.
- ✗
A pre-purchase of tokens at a discounted rate for high-volume Azure OpenAI users
Why it's wrong here
Azure offers commitment tiers or pre-purchased throughput (Provisioned Throughput Units) for cost savings, but that is a commercial arrangement for reserved capacity. The 'token budget' in prompt engineering is the technical constraint of fitting system instructions, conversation history, and document chunks into the model's maximum context length (e.g., 128K tokens for GPT-4 Turbo). Discounts do not change the model's context-window limit or alter the need to manage what content is included.
Go deeper
Related to this question
Learn chapter
Azure AI Language Service
Key term
Model
In IT and AI, a model is a trained mathematical representation that learns patterns from data to make predictions or decisions.
Key term
Token
A token is a digital or physical object that represents a set of permissions or access rights, used to verify identity or grant access in computer systems.
About these practice questions
One of 985 original AI-900 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-900 exam.