What Is Retrieval Augmented Generation (RAG)?
What is 'retrieval augmented generation' (RAG) and which Azure services typically implement it?
Quick Answer
The answer is that retrieval augmented generation (RAG) combines Azure AI Search for retrieval with Azure OpenAI for generation to ground large language model responses in a specific knowledge base. This pattern is correct because it solves a fundamental limitation of generative models: without RAG, an LLM like GPT-4 can only rely on its training data, which may be outdated or incomplete, leading to hallucinations. By first retrieving relevant documents or chunks from a curated knowledge base via Azure AI Search, then passing that context to the Azure OpenAI model, the generated answer is fact-based and verifiable. On the AI-900 exam, this concept tests your understanding of how to make AI outputs trustworthy and domain-specific, often appearing in questions about reducing hallucinations or using enterprise data. A common trap is confusing RAG with fine-tuning—remember that RAG retrieves external data at inference time, while fine-tuning updates the model itself. A helpful memory tip: think of RAG as a librarian handing the right book to a writer, ensuring the story stays accurate.
⚠ Common exam trap
A common mix-up: candidates confuse RAG with fine-tuning, mistakenly thinking retrieval modifies the model's training data, whereas RAG is a prompt-time augmentation that leaves the model unchanged.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Combining Azure AI Search (retrieval) with Azure OpenAI (generation) to ground LLM responses in a knowledge base
Retrieval Augmented Generation (RAG) is a pattern that combines a retrieval step with a generative step. In Azure, this is typically implemented by using Azure AI Search to retrieve relevant documents or chunks from a knowledge base, then passing those results as context to an Azure OpenAI model (e.g., GPT-4) to generate a grounded, fact-based response. This approach reduces hallucinations and ensures the output is based on authoritative data rather than the model's training data alone.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Using Azure Storage to retrieve training data for model fine-tuning
Why it's wrong here
Training data retrieval is part of model fine-tuning — RAG retrieves documents at inference time to provide context for answers.
- ✓
Combining Azure AI Search (retrieval) with Azure OpenAI (generation) to ground LLM responses in a knowledge base
Why this is correct
RAG: AI Search retrieves relevant documents → provided as context to Azure OpenAI → LLM generates answers grounded in retrieved content.
- ✗
Using Azure CDN to deliver AI-generated content faster globally
Why it's wrong here
CDN delivers web content — RAG is an AI architecture for grounding LLM responses in retrieved knowledge.
- ✗
A method of compressing large datasets before training language models
Why it's wrong here
Dataset compression is data engineering — RAG is a runtime architecture for accurate, grounded AI responses.
Go deeper
Related to this question
Learn chapter
Azure Machine Learning Studio
Key term
Training data
Training data is a set of examples used to teach a machine learning model how to make predictions or decisions.
Key term
RAG
Retrieval-Augmented Generation (RAG) is a technique that combines information retrieval with text generation to produce more accurate and contextually relevant answers from large language models.
About these practice questions
One of 985 original AI-900 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
Same concept, more angles
2 more ways this is tested on AI-900
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. What is 'retrieval-augmented generation' (RAG) and what problem does it solve?
medium- A.Storing model responses in a cache to retrieve them faster for repeated questions
- ✓ B.Retrieving relevant documents from a knowledge base to provide accurate context for LLM responses
- C.Generating random responses and selecting the most relevant using a ranker model
- D.A technique for making LLM responses shorter by removing irrelevant sections
Why B: Retrieval-augmented generation (RAG) combines a retrieval step with a generative language model. It first retrieves relevant documents or passages from an external knowledge base (e.g., Azure Cognitive Search) and then feeds that context into the LLM to ground its response. This solves the problem of LLMs producing outdated, hallucinated, or factually incorrect answers by ensuring the model has access to current, authoritative information.
Variation 2. What is the primary benefit of using Retrieval Augmented Generation (RAG) over relying solely on an LLM's trained knowledge?
medium- A.RAG makes LLMs faster by skipping the training process
- ✓ B.RAG grounds LLM responses in current, specific information — reducing hallucination and knowledge cutoff issues
- C.RAG reduces the cost of API calls by batching requests
- D.RAG allows LLMs to process images alongside text
Why B: RAG enhances LLM outputs by retrieving relevant, up-to-date information from an external knowledge base (e.g., Azure Cognitive Search) and injecting it into the prompt context. This grounds the model's response in verifiable data, significantly reducing hallucinations and overcoming the knowledge cutoff limitation inherent in static training data.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-900 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-900 exam.