This chapter covers Question Answering with Azure Language Service, a key capability under the Natural Language Processing (NLP) domain for the AI-900 exam. You will learn how to build and deploy a knowledge base that answers user questions in natural language, which is a core objective for demonstrating understanding of conversational AI. Approximately 5-10% of AI-900 exam questions touch on question answering, focusing on the creation, management, and consumption of a knowledge base using the Language Service.
Jump to a section
Imagine a large library where patrons can ask questions in natural language, like "Who wrote the book on ancient Rome?" The library has a highly trained expert librarian who doesn't just search blindly; instead, she first interprets the question to understand the intent. She then consults a detailed catalog (the knowledge base) that maps topics to specific book titles and authors. The catalog is organized with questions and answers, not just raw text. When a patron asks a new question, the librarian retrieves the most relevant entry from the catalog and formulates a precise answer. If the catalog doesn't have an exact match, she can still provide a confidence-scored answer from the closest entry. The librarian also learns from feedback: if a patron says "That's not what I meant," she notes it and improves future responses. This system works because the catalog is curated and the librarian follows a strict process: parse, match, extract, and respond. In the Azure Language Service, the custom question answering feature works exactly like this librarian: it takes a user query, finds the best matching Q&A pair from a knowledge base, and returns the answer with a confidence score. The knowledge base is built from FAQs, manuals, or documents, and the service can be trained with active learning to improve over time.
What is Question Answering and Why Does It Exist?
Question answering (QA) is a cognitive service that enables applications to respond to user queries with precise answers extracted from a structured knowledge base. In Azure, the Language Service provides a custom question answering feature (formerly QnA Maker) that allows you to import content from FAQ pages, product manuals, support documents, or custom Q&A pairs. The system returns the most relevant answer along with a confidence score, enabling conversational experiences like chatbots or virtual assistants.
How It Works Internally: The Mechanism
When a user submits a query, the Language Service processes it through several steps:
Query Parsing: The service breaks the query into tokens and identifies key phrases using a language model. It normalizes the text (lowercasing, stemming) and removes stop words.
Ranking: The parsed query is compared against all Q&A pairs in the knowledge base using a combination of term frequency-inverse document frequency (TF-IDF) and deep learning models (e.g., BERT-based rankers). Each pair gets a confidence score between 0 and 100.
Answer Selection: The pair with the highest confidence score is selected. If the score is above a configurable threshold (default 0), the answer is returned. If below, a default response like "No good match found" is returned.
Active Learning: The service can monitor user feedback (e.g., thumbs up/down) and suggest alternative question phrasings to the knowledge base curator, improving accuracy over time.
Key Components, Values, Defaults, and Timers
Knowledge Base (KB): A collection of Q&A pairs. Each pair has a question, answer, metadata (e.g., category), and optional follow-up prompts.
Confidence Score: A float from 0 to 100. Default threshold is 0, meaning any match is returned. In practice, set a threshold of 20-50 to filter low-confidence matches.
Language Service Resource: You need a Language resource in Azure (not a QnA Maker resource; QnA Maker is now part of Language Service). Pricing tiers: Free (F0) with limited transactions, Standard (S) with higher limits.
Project: A container for the knowledge base within the Language Service. You create a project of type "Custom Question Answering."
Source Types: You can add sources like URLs (e.g., FAQ pages), files (PDF, DOCX, TXT), or structured Q&A pairs manually.
Synonyms: You can define synonyms to improve matching (e.g., "car" and "automobile").
Multi-turn Prompts: You can chain questions to simulate a conversation (e.g., "What is the return policy?" followed by "How do I return a laptop?").
Configuration and Verification Commands
To create and manage a knowledge base, you use the Azure portal, Language Studio, or REST API. There is no direct CLI command, but you can use Azure CLI to create the Language resource:
az cognitiveservices account create --name myLanguageResource --resource-group myRG --kind TextAnalytics --sku F0 --location westusThen, using the REST API, you can create a project:
curl -X POST "https://myLanguageResource.cognitiveservices.azure.com/language/query-knowledgebases/projects?api-version=2021-10-01" -H "Ocp-Apim-Subscription-Key: {key}" -H "Content-Type: application/json" -d '{"projectName": "myKB", "language": "en", "description": "FAQ KB"}'To query the KB:
curl -X POST "https://myLanguageResource.cognitiveservices.azure.com/language/:query-knowledgebases?projectName=myKB&api-version=2021-10-01" -H "Ocp-Apim-Subscription-Key: {key}" -H "Content-Type: application/json" -d '{"question": "What is your return policy?", "top": 3}'How It Interacts with Related Technologies
Azure Bot Service: The most common use case is to integrate the knowledge base with a bot. The Bot Framework SDK can call the Language Service API to answer user questions.
Cognitive Search: You can combine question answering with Azure Cognitive Search for enterprise search scenarios. The knowledge base acts as a curated Q&A layer on top of indexed documents.
Language Understanding (LUIS): LUIS handles intent classification and entity extraction, while question answering provides direct answers. Together, they power sophisticated conversational AI.
Power Virtual Agents: You can connect a knowledge base to Power Virtual Agents for low-code chatbot creation.
Exam-Relevant Details
The AI-900 exam tests that you can identify when to use custom question answering vs. other NLP services. Key scenario: When you have a set of predefined questions and answers (e.g., FAQ, manual), use custom question answering. If you need to extract answers from unstructured text (e.g., Wikipedia), use the general question answering feature (prebuilt API) or Cognitive Search.
The exam asks about confidence score thresholds and active learning. Remember: active learning suggests alternative questions to improve the KB; it does not automatically update the KB—a human must review and accept suggestions.
You should know that the knowledge base can be published to a public endpoint or used via the Bot Framework.
Common wrong answer: Using LUIS for FAQ answering. LUIS is for intent classification, not for returning specific answers from a fixed set.
Step-by-Step: Creating a Knowledge Base
Create a Language Resource: In Azure portal, create a Language service resource. Choose the Custom Question Answering feature.
Create a Project: In Language Studio, select "Custom question answering" and create a new project. Provide a name and language.
Add Sources: Add a URL (e.g., a FAQ page) or upload a file. The service will automatically extract Q&A pairs.
Edit Pairs: Review and edit the extracted pairs. Add alternate questions, synonyms, and follow-up prompts.
Train and Test: Use the test pane to ask questions and see confidence scores. Adjust threshold as needed.
Publish: Deploy the knowledge base to a public endpoint. You get a URL and key for API access.
Integrate: Use the endpoint in a bot or application.
Performance and Scale Considerations
The Free tier allows up to 3 test documents and 50 transactions per minute. The Standard tier supports production workloads with higher throughput.
Knowledge base size limits: Up to 50,000 Q&A pairs for Standard tier.
Latency: Typically under 2 seconds for a query.
Active learning requires at least 1,000 queries per month to generate meaningful suggestions.
Common Pitfalls
Not setting a confidence threshold: Returns low-quality matches. Always set a threshold (e.g., 30) to filter noise.
Using unstructured text sources: If you upload a long document, the service may not extract good Q&A pairs. Use structured FAQs or manually define pairs.
Ignoring active learning: Without active learning, the KB does not improve over time. Enable it and review suggestions regularly.
Create a Language Resource
In the Azure portal, navigate to 'Create a resource' and search for 'Language service'. Select the Cognitive Service 'Language Service' and click Create. Fill in the subscription, resource group, region, and name. Choose the pricing tier (Free F0 for testing, Standard S for production). Under 'Custom Question Answering', select the features you need. After creation, note the endpoint URL and access key from the 'Keys and Endpoint' section. These credentials are used to authenticate API calls.
Create a Project in Language Studio
Go to Language Studio (https://language.cognitive.azure.com). Select the Language resource you created. Under the 'Custom question answering' tile, click 'Create new project'. Provide a project name (e.g., 'FAQ-KB') and choose the language (e.g., English). Optionally, add a description. The project acts as a container for your knowledge base. You can have multiple projects under one Language resource.
Add Sources to the Knowledge Base
In the project, click 'Add source' to import content. You can add a URL (e.g., 'https://example.com/faq'), upload a file (PDF, DOCX, TXT, XLS), or manually enter Q&A pairs. The service will parse the source and extract question-answer pairs. For URLs, it scrapes the page for FAQ-like content. For files, it uses text extraction. Review the extracted pairs and edit as needed. Each pair can have multiple alternate questions to improve matching.
Test and Train the Knowledge Base
Use the 'Test' pane in Language Studio to simulate user queries. Type a question and see the returned answer along with the confidence score. If the score is low, consider adding alternate questions or adjusting the threshold. You can also provide feedback (thumbs up/down) to improve active learning. Training is automatic; no separate training step is required. However, after editing, you must publish to update the live endpoint.
Publish the Knowledge Base
Click 'Publish' to deploy the knowledge base to a public endpoint. You will get a REST endpoint URL and a key. The endpoint is used by applications to query the KB. Publishing overwrites the previous version. Optionally, you can create multiple versions (e.g., staging, production) by using different projects. After publishing, you can integrate the endpoint with a chatbot using the Bot Framework SDK or direct API calls.
Enterprise Scenario 1: Customer Support FAQ Chatbot
A large e-commerce company wants to reduce support ticket volume by handling common queries via a chatbot. They have a comprehensive FAQ page covering shipping, returns, payment, and account issues. Using Azure Language Service custom question answering, they import the FAQ URL into a knowledge base. They manually add alternate questions based on common user phrasings (e.g., 'How do I return an item?' vs 'What is the return process?'). The chatbot uses the Bot Framework to call the KB endpoint. The confidence threshold is set to 40 to avoid irrelevant answers. Active learning is enabled, and the support team reviews suggestions weekly. In production, the KB handles 10,000 queries per day with a 95% accuracy rate. Common issues arise when users ask multi-part questions (e.g., 'What is the return policy for electronics and how long does it take?'). To handle this, they create follow-up prompts: the first answer returns the policy, and a follow-up prompt asks 'Do you want to know the return timeline?' This requires careful design of multi-turn conversations.
Enterprise Scenario 2: Internal IT Helpdesk
A multinational corporation deploys an internal chatbot for IT support. They have a knowledge base of common IT issues: password reset, software installation, network access. They upload a PDF manual of IT procedures. However, the PDF contains long paragraphs, and the automatic extraction creates poor Q&A pairs. They manually curate the KB by adding specific Q&A pairs for each issue. They also integrate with Azure Active Directory to personalize responses (e.g., 'Your password expires in 10 days'). The chatbot is accessible via Microsoft Teams. Performance considerations: The KB must handle 500 concurrent users. The Standard tier supports this with proper scaling. They set a low confidence threshold (20) because users prefer any answer over 'no match,' but they include a fallback to escalate to a human agent. Misconfiguration: Initially, they did not set a threshold, and the bot returned irrelevant answers with confidence scores as low as 1. After setting the threshold to 20, the bot's accuracy improved.
Enterprise Scenario 3: Healthcare Patient Portal
A hospital creates a patient portal chatbot to answer questions about appointments, billing, and medical records. They use a knowledge base built from structured Q&A pairs created by the administrative team. Because of HIPAA compliance, they deploy the Language Service in a secure region with private endpoints. They also use Azure Cognitive Search to index unstructured medical documents, but for direct Q&A, they rely on the curated KB. The chatbot handles 1,000 queries per day. Common failure: Patients ask questions with medical terminology that is not in the KB (e.g., 'What is a copay?'). The team adds these terms as alternate questions. They also use synonyms (e.g., 'doctor' and 'physician'). The confidence threshold is set to 50 to ensure high accuracy, with a fallback to a human agent for lower scores.
What AI-900 Tests on This Topic
AI-900 objective 4.3: 'Describe question answering capabilities of the Language service.' The exam expects you to know:
The purpose of custom question answering: to provide answers from a knowledge base of predefined Q&A pairs.
How to create a knowledge base: from URLs, files, or manual entry.
The concept of confidence scores and thresholds.
Active learning: how it improves the KB over time.
Integration with Azure Bot Service.
Common Wrong Answers and Why Candidates Choose Them
'Use LUIS for FAQ answering.' Candidates confuse intent classification with direct answering. LUIS identifies the user's intent (e.g., 'GetReturnPolicy') but does not return a specific answer. Custom question answering returns the exact answer.
'Active learning automatically updates the knowledge base.' Candidates think active learning is fully automated. In reality, it only suggests alternative questions; a human must review and accept them.
'Confidence score is a percentage from 0 to 100% and must be set to 0 for best results.' While it is 0-100, setting threshold to 0 returns all matches, including bad ones. The exam tests that you should set a threshold to filter low-confidence answers.
'You can use a PDF to automatically create a perfect knowledge base.' The extraction works but often requires manual editing. The exam expects you to know that manual curation is often needed.
Specific Numbers and Terms on the Exam
Confidence score range: 0-100.
Default threshold: 0 (but recommend 20-50).
Active learning: requires at least 1,000 queries per month for suggestions.
Source types: URL, file (PDF, DOCX, TXT), manual Q&A.
Pricing: Free (F0) and Standard (S).
Multi-turn prompts: used for follow-up questions.
Edge Cases and Exceptions
If the knowledge base has no match, the service returns a default message (e.g., 'No good match found'). You can customize this message.
If two Q&A pairs have the same confidence score, the service returns the first one in the list (order matters).
The service supports multiple languages per project, but you must specify the language at creation.
You can export and import knowledge bases as JSON files.
How to Eliminate Wrong Answers
If the scenario describes a chatbot that needs to answer specific questions from a fixed set, choose custom question answering, not LUIS or Cognitive Search.
If the question mentions 'improving accuracy over time by suggesting new question phrasings,' it refers to active learning.
If the question involves a confidence score, remember that a higher threshold means fewer but more accurate answers.
If the question asks about extracting answers from unstructured text (e.g., Wikipedia), the answer is not custom question answering but the general question answering API or Cognitive Search.
Custom question answering returns answers from a knowledge base of Q&A pairs, not from unstructured text.
Confidence score ranges from 0 to 100; set a threshold (e.g., 30) to filter low-quality matches.
Active learning suggests alternative question phrasings but requires human approval to update the KB.
Sources can be URLs, files (PDF, DOCX, TXT), or manual entry.
The Free tier (F0) is limited; use Standard (S) for production.
Multi-turn prompts enable follow-up questions in a conversation.
Integrate with Azure Bot Service for chatbot deployment.
These come up on the exam all the time. Here's how to tell them apart.
Custom Question Answering
Returns a specific answer from a knowledge base.
Best for FAQ-style queries with predefined answers.
Uses confidence scores to rank matches.
Supports multi-turn conversations with follow-up prompts.
Integrated with Azure Bot Service for chatbots.
Language Understanding (LUIS)
Returns an intent and entities, not a direct answer.
Best for understanding user goals and extracting parameters.
Uses scores for intent classification, not answer matching.
Supports multi-turn via context, but no predefined answers.
Also integrated with Azure Bot Service, but for different use cases.
Custom Question Answering
Answers from a curated knowledge base of Q&A pairs.
Returns a single answer with confidence score.
Designed for conversational Q&A.
No indexing of unstructured text; relies on predefined pairs.
Active learning improves question matching over time.
Azure Cognitive Search
Returns ranked search results from indexed documents.
Returns a list of documents with relevance scores.
Designed for enterprise search across large datasets.
Indexes unstructured text and extracts answers using AI.
No built-in active learning for Q&A improvement.
Mistake
Custom question answering can answer any question from any document.
Correct
It only answers questions that have a matching Q&A pair in the knowledge base. It does not reason or generate new answers; it retrieves predefined answers.
Mistake
Active learning automatically updates the knowledge base without human intervention.
Correct
Active learning only suggests alternative question phrasings. A human must review and accept these suggestions to update the KB.
Mistake
You must use the Bot Framework to consume a question answering endpoint.
Correct
You can use the REST API directly from any application. The Bot Framework is one integration option, not a requirement.
Mistake
The confidence score is a percentage that indicates how correct the answer is.
Correct
It is a relative score indicating how well the query matches a Q&A pair compared to others. It does not measure factual correctness.
Mistake
You can only create a knowledge base from FAQ URLs.
Correct
You can also upload files (PDF, DOCX, TXT) or manually enter Q&A pairs. URLs are one of several source types.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
Custom question answering requires you to build a knowledge base of Q&A pairs. The general question answering API (prebuilt) uses a built-in model to extract answers from unstructured text like Wikipedia. Use custom when you have specific, curated content; use prebuilt for general knowledge queries.
In the project settings in Language Studio, you can set a confidence score threshold. Alternatively, when calling the REST API, you can specify a 'confidenceScoreThreshold' parameter. The default is 0, meaning all matches are returned. Set it to a value like 30 to filter low-confidence answers.
No, any changes to the knowledge base (adding/editing Q&A pairs) require publishing to make them live. Until you publish, the test environment reflects changes, but the production endpoint uses the last published version.
It supports over 50 languages. When creating a project, you specify the language. The service uses language-specific models for parsing and ranking. You can have separate projects for different languages.
Active learning monitors user queries that return low-confidence answers. It groups similar queries and suggests them as alternate questions. A human curator reviews these suggestions in the Language Studio and can accept or reject them. Accepted suggestions are added to the KB.
For the Standard tier, the maximum number of Q&A pairs is 50,000. The Free tier allows up to 3 test documents and limited pairs. There is also a limit on the total size of the KB in terms of storage (e.g., 10 GB for Standard).
Yes, you can call the REST API directly from any application (web, mobile, etc.). The endpoint returns JSON with the answer and confidence score. You don't need a bot framework.
You've just covered Question Answering with Language Service — now see how well it sticks with free AI-900 practice questions. Full explanations included, no account needed.
Done with this chapter?