Reducing Hallucinations with Retrieval-Augmented Generation
An organization is deploying a conversational AI solution using Azure OpenAI. They want to ensure the model's responses are grounded in their own knowledge base documents to reduce hallucinations. Which approach should they implement?
Quick Answer
The correct approach is to integrate Azure Cognitive Search for retrieval-augmented generation (RAG). This grounds Azure OpenAI with RAG to reduce hallucinations by dynamically retrieving relevant chunks from the organization’s knowledge base at inference time, providing factual context instead of relying solely on the model’s parametric memory. On the Microsoft Azure AI Engineer Associate AI-102 exam, this scenario tests your understanding of how RAG combines a vector or keyword search index with a generative model to anchor responses in authoritative, up-to-date content. A common trap is choosing fine-tuning alone, which updates the model’s weights but does not reference live documents, leaving hallucinations possible. Remember the mnemonic “Search before Generate” — always pair Azure Cognitive Search with OpenAI to ground answers, ensuring the model cites your data rather than guessing.
⚠ Common exam trap
Many exam-takers confuse fine-tuning (B) as a way to 'teach' the model the knowledge base, not realizing that RAG is the recommended pattern for grounding responses in external, query-specific data without retraining.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Integrate Azure Cognitive Search for retrieval-augmented generation (RAG)
Retrieval-Augmented Generation (RAG) with Azure Cognitive Search allows the model to dynamically retrieve relevant chunks from the organization's knowledge base documents at inference time. This grounds responses in authoritative, up-to-date content, directly reducing hallucinations by providing factual context rather than relying solely on the model's parametric memory.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Integrate Azure Cognitive Search for retrieval-augmented generation (RAG)
Why this is correct
RAG with Cognitive Search grounds responses in retrieved documents, reducing hallucinations.
- ✗
Fine-tune the model on the knowledge base documents
Why it's wrong here
Fine-tuning can teach the model about the domain but does not guarantee factual accuracy for all queries.
- ✗
Implement Azure AI Content Safety filters
Why it's wrong here
Content Safety filters harmful content but does not ground responses in factual data.
- ✗
Use prompt engineering to instruct the model to only use the knowledge base
Why it's wrong here
Prompt engineering alone is insufficient to prevent the model from inventing information.
Go deeper
Related to this question
About these practice questions
This AI-102 question is part of Courseiva's 945-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on AI-102
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A company uses Azure OpenAI Service to generate product descriptions. They notice that the descriptions sometimes contain factually incorrect information. Which strategy should they use to reduce hallucinations?
medium- A.Increase the temperature parameter to 1.0.
- ✓ B.Implement Retrieval-Augmented Generation (RAG) by grounding prompts with a knowledge base.
- C.Reduce the max_tokens parameter to limit output length.
- D.Add a system message instructing the model to be more careful.
Why B: Retrieval-Augmented Generation (RAG) grounds the model's output in a trusted, external knowledge base, providing factual context that directly reduces hallucinations. By retrieving relevant documents and injecting them into the prompt, the model generates responses based on verified information rather than relying solely on its parametric memory, which is the primary cause of factual inaccuracies in Azure OpenAI Service.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI-102 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI-102 exam.