AI-900 Practice Question: Describe features of generative AI workloads on Azure
What is 'Azure OpenAI's batch API' and when should you use it?
⚠ Common exam trap
Candidates often confuse batch processing for inference with batch training of models, leading them to select Option A, but Azure OpenAI's Batch API is strictly for inference, not model training.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Asynchronous bulk processing of large inference request volumes at reduced cost
Azure OpenAI's Batch API is designed for asynchronous processing of large volumes of inference requests, such as chat completions or embeddings, at a reduced cost compared to real-time API calls. It is ideal for workloads where immediate responses are not required, allowing you to submit a batch of requests and retrieve results later. This makes it a cost-effective solution for high-throughput, non-latency-sensitive tasks.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
An API for training new models in batches on your custom datasets
Why it's wrong here
Training custom models on your own datasets is a distinct capability called fine-tuning, where you submit paired instruction-response examples via the Azure OpenAI fine-tuning API to update model weights. The Batch API, in contrast, only sends prompt requests for inference and returns completions—it never ingests training labels or modifies the underlying model. Referring to 'batch training' here is a category error: in Azure OpenAI, batch denotes bulk asynchronous inference, not batch gradient descent or supervised fine-tuning.
- ✓
Asynchronous bulk processing of large inference request volumes at reduced cost
Why this is correct
The Azure OpenAI Batch API is purpose-built for high-volume, asynchronous inference workloads, accepting thousands of prompts in a single JSONL file and processing them within a 24-hour window. This offline execution model delivers roughly 50% cost savings compared to real-time pay-as-you-go calls, making it ideal for tasks like bulk document summarization or data extraction. It decouples throughput from latency, so you send the batch, poll for status, and retrieve results from a designated output file.
- ✗
Grouping multiple Azure OpenAI API keys into a batch for easier management
Why it's wrong here
API key management in Azure involves Azure resource administration (IAM roles, access control, and Key Vault for secret storage), and it has no connection to the Batch API's function. Grouping keys into a 'batch' would be meaningless because each batch job already runs under a single resource endpoint and credential, with no need for multi-key orchestration. This option likely misinterprets the word 'batch' as 'a collection of credentials,' whereas the Batch API is about batching inference requests, not batching keys.
- ✗
A tool for running multiple prompt experiments simultaneously to find the best prompt
Why it's wrong here
Prompt experimentation and optimisation are performed through evaluation tools like Azure AI Studio's prompt flow, which runs multiple prompt variants and scores them against test data—not by the Batch API. The Batch API does not facilitate interactive A/B testing or hyperparameter search; it simply executes a pre-defined prompt file at scale. Confusing the service with a 'batch of experiments' misreads its role: it is a production inference pipeline, not a development-time prompt-tuning harness.
Go deeper
Related to this question
About these practice questions
One of 985 original AI-900 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-900 exam.