Knowledge Mining — it solves the problem of finding the needle of information in the haystack of your business documents. For the AI-102 exam, you need to know how to use Azure’s Document Intelligence service (formerly known as Form Recognizer) to automatically read, extract, and structure data from invoices, receipts, business cards, and custom forms. This is the engine that powers everything from automated expense reporting to insurance claim processing.
Jump to a section
A simple way to picture Knowledge Mining with Azure Document Intelligence
Have you ever had to find a single receipt in a mountain of crumpled paper receipts stuffed into a shoebox?
That shoebox is your company‘s document archive. You know that receipt is in there somewhere — it holds the key to proving a shipment arrived, or to getting a refund for a damaged item. But digging through every single scrap of paper, one by one, is maddeningly slow, and your eyes start to glaze over after the first hundred.
Now imagine you could take that whole shoebox and dump it into a magical sorting machine. You tell the machine: “I need every receipt that has the word ‘Widget’ in the description, was dated between January and March, and was signed by a person named ‘Jane Smith’.” The machine instantly fans out all the receipts, reads them without getting tired, and then, in the blink of an eye, slides a neat stack across to you — only the receipts that match your search. It has extracted the key information from each one: the date, the item name, the signatory, even the total amount. It has turned your chaotic pile of paper into a searchable, structured database. That is exactly what Knowledge Mining with Azure Document Intelligence does for your business documents. It takes unstructured data (like scanned PDFs, images, or typed forms) and magically organises it so you can ask questions and get precise answers, without ever needing to look at the documents yourself.
At its simplest, Knowledge Mining is the process of turning raw, unstructured data into searchable, actionable knowledge. Azure Document Intelligence is a specific Azure service that performs one of the most common and difficult mining tasks: extracting information from documents.
Let‘s break down the key terms.
Unstructured data is information that doesn’t have a predefined model or organisation. A PDF of an invoice is unstructured — the invoice number might be in the top-right corner on one document and in the bottom-left on another. The data is there, but a computer program can‘t easily find it without being taught where to look. Structured data, on the other hand, is like a spreadsheet or a database table — every piece of information has its own column, and the computer knows exactly where to read it.
Azure Document Intelligence is a cloud-based service that uses pre-built models and custom-trained models to extract structured data from documents. Think of it as a super-smart scanner that doesn’t just take a picture — it actually reads and labels the information. The service is built on Azure, Microsoft’s cloud platform, which means you don’t need to install any software or manage any servers. You just send your documents via an API (Application Programming Interface, a way for different programs to talk to each other), and you get back a structured JSON (JavaScript Object Notation, a lightweight data-interchange format that is easy for computers to read) file containing all the extracted fields.
The service works through a combination of optical character recognition (OCR) and machine learning models. OCR is the technology that converts images of text into machine-readable text — it’s how the service can “read” a scanned piece of paper. The machine learning models are trained on millions of documents (like invoices, receipts, and business cards) to recognise patterns. For example, a model might learn that the word “Total” followed by a number in a particular font is almost always the total amount due.
Why does this exist? Because manual data entry is slow, expensive, and error-prone. Instead of paying someone to type invoice numbers and line items into a database, you can have Document Intelligence do it automatically in seconds. This frees up human workers to focus on higher-value tasks, like resolving exceptions and building customer relationships.
Now, the AI-102 exam focuses on how to implement a knowledge mining solution. This involves several components:
Choosing the right model type. There are pre-built models for common document types like invoices, receipts, identity documents, and business cards. There is also a custom model option, where you train the service on your own specific form types, like a unique job application form or a custom purchase order.
Calling the REST API (a standardised way to communicate over the internet) or using a client library SDK (Software Development Kit, a collection of tools and libraries to help you code). You need to know the correct HTTP methods (like POST to submit a document) and how to handle the asynchronous response (meaning the service takes time to analyse the document and then sends back the result).
Understanding the response. The JSON output includes confidence scores for each extracted field — a number between 0 and 1 that tells you how sure the model is about that particular extraction. If the confidence score is low, you might need to manually verify that field.
Analysing the output with Azure AI Search. Document Intelligence is often paired with Azure AI Search to build a full knowledge mining solution. Once you have extracted structured data, you index it (make it searchable) so that users can run natural language queries like “Show me all invoices from Acme Corp with totals over £1000.” This combination is the heart of knowledge mining.
The service replaces manual data entry, dedicated OCR software that requires specialised training, and expensive third-party document processing tools. It brings the power of AI to everyday business processes.
Create an Azure Document Intelligence Resource
In the Azure portal, you provision a Document Intelligence resource. This allocates compute capacity and gives you an endpoint URL and authentication key. This is the entry point to using the service.
Choose Your Model Type (Pre-built or Custom)
Analyse your document type. If it is a standard invoice, use the pre-built invoice model. If it is a unique form, you will need to train a custom model. This decision determines your next step.
Submit the Document via the REST API or SDK
You send an HTTP POST request to the Document Intelligence endpoint, attaching the document (PDF, TIFF, JPEG, or PNG). For single-page documents, you can use the synchronous API. For multi-page documents, you must use the asynchronous API.
Poll for the Analysis Result (if asynchronous)
If you used the asynchronous API, the initial response contains an operation-location header with a URL. Your application pings (polls) that URL every few seconds until the status is ‘succeeded’. This ensures you get the complete result without blocking your application.
Parse the JSON Response and Handle Confidence Scores
The service returns a JSON object with fields like ‘fields’ containing key-value pairs, each with a ‘value’ and ‘confidence’. Write logic to check confidence scores: if below your threshold (e.g., 0.9), flag that field for manual review. This is essential for maintaining data quality.
Store the Extracted Data in a Structured Location
The extracted data is now in a structured JSON format. You can insert it into a database, send it to an ERP system via API, or index it in Azure AI Search for full-text querying. This completes the knowledge mining pipeline.
Let’s walk through a realistic scenario. Sarah works as a systems engineer for a large logistics company called “GlobalShip.” The company processes thousands of shipping invoices from carriers every month. Each invoice is a multi-page PDF with line items for every shipment. Currently, a team of ten data entry clerks spends three weeks each month manually typing the invoice details into the company’s accounting software.
Sarah has been tasked with automating this process using Azure Document Intelligence. Here is the step-by-step walkthrough of what an IT professional like Sarah actually does.
First, Sarah navigates to the Azure portal — the web-based management console for Azure services. She creates a new Document Intelligence resource, which is essentially a container for her service. She gives it a name, selects the pricing tier (there is a free tier for testing and paid tiers for production), and chooses her Azure region (like West Europe). The Azure portal then gives her two critical pieces of information: the endpoint (the URL where her service lives) and a key (a secret password that authenticates her requests).
Next, she analyses the invoices. She knows that GlobalShip’s invoices have a standard layout but include some custom fields like the depot reference number. She decides to start with the pre-built invoice model, which should handle the standard fields like invoice number, date, vendor name, and total amount. She writes a short script in Python using the Azure Document Intelligence SDK. The script does the following:
It sends a POST request to the API endpoint, attaching a sample PDF invoice as the document to analyse.
The API responds with an operation-location header, which is a URL she can poll (check repeatedly) to get the analysis result. Because document analysis can take a few seconds, it is implemented as an asynchronous operation.
Sarah’s script then polls that URL every second until the status changes from “running” to “succeeded.” When it succeeds, she receives a JSON response containing all the extracted fields, each with a value and a confidence score.
Sarah examines the JSON output. She notices that the pre-built model correctly extracted the invoice number (confidence: 0.99), the vendor name (confidence: 0.95), and the total (confidence: 0.97). However, the depot reference number was not extracted because it is not a standard field in the pre-built model. To capture this custom field, she will need to train a custom model.
To train a custom model, Sarah gathers about ten sample invoices that all share the same layout. She uploads them to a storage container in Azure Blob Storage (a service for storing large amounts of unstructured data). Then, using the Document Intelligence Studio — a graphical tool that lets you train models without coding — she labels the fields she wants to extract. She draws a bounding box around the depot reference number on each sample and labels it “DepotRef.” The service learns that on these forms, the area below the shipping date and to the left of the total is always the depot reference. After training, she can apply this custom model to the remaining invoices.
Finally, Sarah integrates the extracted data. Her script now sends each invoice to Document Intelligence, receives the JSON output, and then pushes that data directly into the accounting software’s API. The data entry clerks are reassigned to higher-value work, like verifying flagged invoices where confidence scores are low, and the month-end close is completed in two days instead of three weeks.
For a complete knowledge mining solution, Sarah also indexes the extracted data in Azure AI Search. This allows managers to query historical invoices using natural language, such as “What were our total shipping costs from DHL in Q4?” The search index can also include the original PDF for easy reference.
The AI-102 exam tests your understanding of Azure Document Intelligence under the broader objective of implementing knowledge mining solutions. You can expect approximately 5-10% of the exam questions to relate directly to this topic. The questions are a mix of scenario-based single-answer questions, multiple-choice, and sometimes drag-and-drop to order steps. Here is exactly what they test.
First, they focus on model selection. A typical question gives you a description of a business need — for example, “A company needs to extract data from a custom repair order form that has a unique layout not found in any standard model.” The correct answer is always “Use a custom model.” The traps involve suggesting a pre-built model or a different service entirely. They love testing your ability to distinguish between pre-built (for invoices, receipts, ID documents, business cards) and custom (for any form with a consistent layout).
Second, they test your understanding of the synchronous vs asynchronous API. The biggest trap here is thinking that a synchronous operation can handle large documents. The truth is that analysis of a single page is synchronous (you send the document and get the result immediately in the same response), but analysis of multi-page documents is asynchronous (you get a URL to poll for results). Questions will present a scenario with a large PDF and ask you to choose the correct method. The answer is always asynchronous for documents that have more than one page.
Third, they test confidence scores. You will see questions like “When should a human review the extracted data?” The correct answer is “When the confidence score for a field is below a defined threshold, typically below 0.8 or 0.9 depending on the business requirement.” The trap is to think that you should always review when the confidence is 1.0 — no, that is perfect confidence. You review when it is low.
Fourth, they test the integration with Azure AI Search. You need to know that Document Intelligence provides the extraction layer, and Azure AI Search provides the indexing and querying layer. Together they form a knowledge mining solution. Questions may ask: “Which two services are needed to build a complete knowledge mining solution?” Answer: Azure Document Intelligence and Azure AI Search. The trap is to suggest Azure Cognitive Services (a broader category) or Azure OpenAI Service (for generative AI).
Fifth, they test the types of output. The service returns structured JSON. They ask: “What format does Document Intelligence return?” Answer: JSON. The trap is “XML” or “CSV.” You can later convert to CSV, but the native output is JSON.
Sixth, they test identity documents. There is a specific pre-built model for ID documents (like passports and driving licences). They love to test that this model can extract fields like date of birth, document number, and expiration date. A question might say: “Which pre-built model should you use to extract an employee’s date of birth from their passport scan?” Answer: the ID document pre-built model.
Finally, they test licensing and pricing. Be aware that there is a free tier (S0) for up to 500 pages per month. Questions on pricing are rare but can appear.
Key definitions to memorise:
Pre-built models: Invoice, Receipt, ID Document, Business Card, and the newer models like Contract, Tax Document, etc. For the exam, focus on Invoice, Receipt, ID Document, and Business Card.
Custom model: An extraction model trained on your own documents. Requires at least five samples of the same layout.
Asynchronous operation: Used for multi-page documents. Returns an operation-location header that you poll until completion.
Confidence score: A value between 0 and 1 indicating the model’s certainty in the extracted value. Low confidence means human review is needed.
Azure AI Search: The service that indexes extracted data and enables full-text search and filtering.
Document Intelligence Studio: The graphical interface for training custom models without writing code.
Studio vs REST API vs SDK: You can use any of these to interact with the service. The exam doesn’t require you to write code, but you need to understand the capabilities of each approach.
Azure Document Intelligence (formerly Form Recognizer) extracts structured data like key-value pairs, tables, and text from documents using pre-built or custom models.
The service returns confidence scores for every extracted field, and you should always set a threshold to flag low-confidence fields for human review.
Pre-built models exist for invoices, receipts, identity documents, and business cards — use them before considering a custom model.
To build a complete knowledge mining solution, you must pair Document Intelligence with Azure AI Search for indexing and querying.
Custom models require at least five sample documents with the same layout, and you can retrain them if the form design changes.
Document analysis for multi-page documents is asynchronous — you poll a URL until the service returns a ‘succeeded’ status.
These come up on the exam all the time. Here's how to tell them apart.
Azure Document Intelligence (Form Recognizer)
Extracts structured data from documents like invoices and forms.
Returns JSON output with key-value pairs, tables, and confidence scores.
Focuses on data extraction, not indexing or querying.
Azure AI Search
Indexes structured and unstructured data for full-text search.
Allows users to query indexed data using natural language or filters.
Focuses on making data searchable and retrievable, not on extraction.
Pre-built Model
Trained by Microsoft on millions of common documents (invoices, receipts).
No training required from the user; works immediately.
Limited to specific document types that Microsoft covers.
Custom Model
Trained by the user on their own document samples.
Requires at least five sample documents with the same layout.
Can be adapted to any custom form or document layout.
Synchronous API Call
Sends the document and receives the result in the same HTTP response.
Only works for single-page documents.
Faster for simple use cases but limited by document size.
Asynchronous API Call
Sends the document and gets back a URL to poll for the result.
Required for multi-page documents (any document with more than one page).
Handles larger payloads and longer processing times.
Document Intelligence Studio
Graphical web interface for testing and training models.
No coding required; ideal for non-developers to explore.
Limited in automation and batch processing capabilities.
REST API / SDK
Programmatic interface for integrating into applications.
Requires coding in languages like Python, C#, or JavaScript.
Allows full automation, batch processing, and custom logic.
Mistake
Azure Document Intelligence can extract data from any document, even handwritten notes with no structure.
Correct
The service is optimised for structured or semi-structured documents like forms, invoices, and receipts. While it can read handwritten text via OCR, it is not designed to extract meaning from free-form handwritten letters or complex drawings without a custom model trained on that specific layout.
People see the word ‘intelligence’ and assume it is a general-purpose AI that can handle anything, like a human brain. The reality is that it works best when the document layout is consistent and the information follows a predictable pattern.
Mistake
You must use a custom model for every task, because pre-built models are too limited.
Correct
Pre-built models cover the most common business documents (invoices, receipts, identity documents, business cards). For over 80% of typical use cases, a pre-built model is sufficient and requires no custom training. Custom models are only needed for unique forms that are not covered by pre-built models.
Beginners often overestimate the complexity of AI and assume everything requires custom training. They do not realise Microsoft has already trained models on millions of documents, saving them the effort.
Mistake
The output from Document Intelligence is always 100% accurate, so you can trust it completely and never review it.
Correct
Document Intelligence provides confidence scores for each extracted field. These scores indicate how certain the model is. Accuracy is never 100%; there is always a risk of misreading a number or misinterpreting a label. Best practice is to set a confidence threshold (e.g., 0.9) and flag any field below that for human review.
Beginners come from a world of deterministic software (like a calculator that always gives the same answer for 2+2). AI is probabilistic — it gives its best guess, not a guaranteed correct answer. This difference is fundamental to understand.
Mistake
Document Intelligence and Azure AI Search are the same service; you only need one to do knowledge mining.
Correct
They are two separate services that work together. Document Intelligence is for extracting structured data from documents. Azure AI Search is for indexing that data and making it searchable by users. A complete knowledge mining solution uses both.
Both services appear under the ‘AI + Machine Learning’ umbrella in Azure. Beginners assume that one service can do everything, but Microsoft deliberately separates the extraction layer from the search indexing layer for scalability and flexibility.
Mistake
Once you train a custom model, you never need to train it again — it will work on any new version of the form.
Correct
If the form layout changes — for example, a company redesigns its invoice — the custom model will likely fail on the new version. You need to retrain the model on the new layout, or use a technique called ‘composed models’ where you combine multiple custom models trained on different layouts.
People assume AI is ‘set it and forget it.’ In reality, models degrade when the input data distribution changes (a concept called data drift). Beginners often miss this nuance.
Mistake
You can only use Document Intelligence through the Azure portal; you cannot integrate it into an application.
Correct
Document Intelligence is designed for API integration. You call it via REST endpoints or SDKs (Python, C#, Java, JavaScript). The Azure portal provides a studio for testing and training, but production use always involves calling the API from within your own application.
Non-developers often think of Azure tools as standalone web apps. They don’t realise the core value is programmatic access, which allows automation.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
Yes, Form Recognizer was the original name of the service. Microsoft rebranded it to Azure Document Intelligence, but you will still see the old name in some documentation. For the AI-102 exam, they use the new name.
Yes, the OCR engine within Document Intelligence can read handwritten text, but the accuracy depends on the legibility of the handwriting. It works better on printed text. For handwritten forms, test your specific case and review low-confidence outputs.
No. If your document matches a pre-built model category (invoice, receipt, ID card, business card), you do not need any training. Only train a custom model if your document layout is unique and not covered by pre-built models.
It supports JPEG, PNG, TIFF, BMP, and PDF. For PDFs, ensure they are not password-protected. The service works best with high-resolution, clear documents.
Microsoft recommends at least five samples of the same form layout for training. The more samples you provide (up to 50 or more), the higher the accuracy of your custom model will be.
Yes, the pre-built models and custom models both support table extraction. The response returns tables as an array of rows and cells with their content and confidence scores. This is especially useful for invoices with line-item tables.
You've finished Knowledge Mining with Azure Document Intelligence. Continue through the AI-102 study guide to build a complete picture of the exam.
Done with this chapter?