This chapter covers Microsoft Copilot, a generative AI tool integrated into Microsoft 365 and Azure. For the AI-900 exam, understanding Copilot's architecture, capabilities, and limitations is essential as it represents a key application of generative AI in enterprise. Approximately 10-15% of exam questions touch on Copilot, focusing on its responsible AI principles, data handling, and integration with Microsoft services. You'll learn how Copilot works under the hood, its key components like grounding and content filtering, and how it differs from other AI services.
Jump to a section
Microsoft Copilot is like having a highly knowledgeable but strictly supervised intern in a library. This intern has read every book in the library (the entire internet and Microsoft documentation up to a certain date) but is only allowed to answer questions using the books you specifically hand them (your data). They cannot make up facts; they must quote directly from the provided books. However, they are skilled at summarizing, rephrasing, and connecting ideas from those books. If you ask a question outside the books, they politely decline or say they don't know. The intern also keeps a log of every interaction, which you can review to ensure they didn't violate any rules. Importantly, the intern never learns from your questions to teach another person; each conversation is isolated. This mirrors how Copilot works: it uses a large language model (the intern's brain) but grounds its responses in your specific data (the books), uses content filtering to block harmful outputs, and preserves privacy by not using your prompts for training.
What is Microsoft Copilot?
Microsoft Copilot is a generative AI assistant that combines large language models (LLMs) with your organization's data and Microsoft 365 apps. It is not a single product but a family of copilots: Copilot for Microsoft 365 (for Office apps), GitHub Copilot (for code), and Azure AI Copilot (for cloud management). All share a common architecture: they use an LLM, ground responses in relevant data, apply safety filters, and orchestrate actions via plugins.
How Copilot Works Internally
Copilot's process involves four key steps: 1. Prompt processing: The user enters a natural language prompt (e.g., 'Summarize last week's sales report'). 2. Grounding: Copilot retrieves relevant data from Microsoft Graph (emails, documents, calendar) or Azure resources. This data is injected into the prompt as context. 3. LLM inference: The grounded prompt is sent to an LLM (likely GPT-4 or a variant) hosted in Azure. The model generates a response based on the context. 4. Post-processing: The response passes through content filters (for harmful content), responsible AI checks, and formatting. The final answer is returned to the user.
Key Components and Values
Microsoft Graph: The data source for Copilot for Microsoft 365. It indexes user's emails, files, calendar, and contacts. Copilot only accesses data the user has permissions to view.
Grounding: The process of adding retrieved data to the prompt. This reduces hallucinations because the model must answer based on provided context.
Plugins: Extend Copilot's capabilities (e.g., connect to Jira, ServiceNow). Plugins are governed by the same security and compliance policies.
Content filtering: Built-in filters block hate speech, violence, self-harm, and sexual content. They operate in real-time on both input and output.
Data residency: Copilot processes data in the region where the tenant is located. Data is not used to train the underlying LLM.
Usage limits: For Copilot for Microsoft 365, there are per-user daily message limits (e.g., 100 messages per day per user) to prevent abuse.
Configuration and Verification
Administrators configure Copilot in the Microsoft 365 admin center. Key settings include: - Data access: Which SharePoint sites, OneDrive folders, or mailboxes Copilot can search. - Plugin management: Enable or disable third-party plugins. - Audit logging: All Copilot interactions are logged in the Microsoft 365 audit log for compliance.
To verify Copilot is working, an admin can check the audit log with PowerShell:
Search-UnifiedAuditLog -StartDate (Get-Date).AddDays(-1) -Operations CopilotInteractionThis returns all Copilot interactions in the last 24 hours.
Interaction with Related Technologies
Copilot integrates with: - Azure OpenAI Service: Underlying LLM infrastructure. Copilot uses Azure OpenAI for inference but abstracts away the model details. - Microsoft 365: Copilot appears as an add-in in Word, Excel, PowerPoint, Teams, and Outlook. - Power Platform: Copilot can be used in Power Automate to generate flows or in Power Virtual Agents to create chatbots. - Azure AI Search: For custom copilots (Azure AI Copilot), grounding uses Azure AI Search to index enterprise data.
Responsible AI and Copilot
Microsoft applies six responsible AI principles to Copilot: fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability. Concrete implementations include: - Transparency: Copilot indicates it is an AI and provides citations to sources. - Privacy: Prompts and responses are not used to train the base model. Data is encrypted in transit and at rest. - Safety: Content filters prevent generation of harmful content. Human review is used for abuse detection.
Limitations
Copilot can only access data that the user has permission to see. It cannot bypass security boundaries.
It may produce plausible-sounding but incorrect information (hallucinations), especially if grounding data is insufficient.
It has no memory of past conversations in the same session unless explicitly designed (e.g., in Teams chat).
It is limited to the training data cutoff date (e.g., early 2023 for some models) unless grounded with fresh data.
AI-900 Exam Focus on Copilot
The exam tests your understanding of: - What Copilot is: A generative AI assistant that combines LLMs with user data. - How it grounds responses: By retrieving data from Microsoft Graph or Azure AI Search. - Responsible AI features: Content filtering, privacy (no training on user data), transparency (citations). - Comparison with other services: Copilot vs. Azure OpenAI Service vs. ChatGPT.
Common exam scenarios:
A company wants to use AI to help employees draft emails based on past correspondence. Which service? → Copilot for Microsoft 365.
A developer wants to generate code suggestions. → GitHub Copilot.
An admin wants to ensure AI doesn't access sensitive data. → Copilot respects user permissions via Microsoft Graph.
User Inputs Prompt
The user types or speaks a natural language prompt in a Copilot-enabled application (e.g., Word, Teams). The prompt is captured and sent to the Copilot backend. This step includes any context from the current document or conversation (e.g., the document content in Word). The system also collects metadata like user identity and permissions.
Grounding Data Retrieval
Copilot queries Microsoft Graph (for M365) or Azure AI Search (for custom copilots) to find relevant data. It uses semantic search to identify documents, emails, or files that match the user's intent. The retrieved data is appended to the prompt as context. Only data the user has permission to access is included. The number of retrieved documents is typically limited to 10-20 to keep the prompt within token limits.
LLM Inference
The grounded prompt is sent to an LLM (e.g., GPT-4) hosted on Azure. The model generates a response token by token. The inference uses the context to answer accurately. The model's parameters (temperature, top_p) are set for factual responses (low temperature ~0.3). The inference time is typically 1-5 seconds depending on response length.
Content Filtering
The generated response passes through Azure AI Content Safety filters. These check for hate speech, violence, self-harm, and sexual content. If the response is flagged, it is either blocked entirely or modified (e.g., replacing harmful words). The filters operate on both input and output. Administrators can set severity thresholds (low, medium, high) to control sensitivity.
Response Delivery
The filtered response is returned to the user's application. In Word, it appears as a suggested edit; in Teams, as a chat message. The response includes citations to the sources used (e.g., file names). The entire interaction is logged in the audit log for compliance. The user can provide feedback (thumbs up/down) to improve the system.
Enterprise Scenario 1: Automating Email Drafts in a Sales Department
A global sales team uses Copilot for Microsoft 365 to draft personalized emails to prospects. The problem: sales reps spend 2 hours daily writing emails. Solution: In Outlook, the rep clicks 'Draft with Copilot' and enters a prompt like 'Write a follow-up email to John about our meeting last week, mentioning the discount we discussed.' Copilot retrieves the user's calendar and emails to find the meeting details and previous correspondence. It then generates a draft that the rep can edit. Configuration: Admin enabled Copilot for all sales users, granted access to SharePoint site with product info, and set content filter to medium severity. Performance: Each draft takes ~3 seconds. Misconfiguration: If the admin forgets to index the SharePoint site, Copilot cannot ground responses with product details, leading to generic drafts that miss key information. This causes reps to manually edit more, reducing time savings.
Enterprise Scenario 2: IT Support Chatbot with Azure AI Copilot
A large enterprise wants an internal chatbot to answer IT support questions. They use Azure AI Copilot (custom copilot) with Azure AI Search indexing their knowledge base (SharePoint, Confluence). The problem: IT support tickets take 24 hours to resolve. Solution: Employees ask the chatbot 'How do I reset my password?' and get step-by-step instructions. The chatbot is deployed in Microsoft Teams. Configuration: The knowledge base is indexed daily. The copilot uses GPT-4 with grounding from Azure AI Search. Content filters are set to high to block any sensitive info. Performance: Response time is under 2 seconds. Common failure: If the index is stale, the chatbot gives outdated instructions (e.g., old password policy). Administrators must monitor index freshness and set up alerts for failed indexing.
Enterprise Scenario 3: Code Generation in Development Teams
A software company uses GitHub Copilot to accelerate coding. Developers get inline code suggestions in VS Code. The problem: developers spend 30% of time writing boilerplate. Solution: GitHub Copilot suggests code as they type. Configuration: The company uses a business license, which provides admin controls to disable Copilot for certain repositories (e.g., those with sensitive code). Performance: Suggestions appear in <1 second. Misconfiguration: If a developer accidentally includes proprietary code in a prompt, that code is sent to GitHub's servers. Although GitHub Copilot does not store prompts, the company's compliance team must ensure developers are trained not to paste sensitive data. The admin can block specific domains or use telemetry to review usage.
AI-900 Objective Codes Tested
5.4 Describe generative AI: Specifically, 'Describe Microsoft Copilot' and 'Identify use cases for Copilot.' The exam expects you to know what Copilot is, how it differs from ChatGPT, and its responsible AI features.
Common Wrong Answers and Why
'Copilot uses the same model as ChatGPT' – Wrong. While both may use GPT-4, Copilot grounds responses in your data and applies content filters. ChatGPT does not have grounding or enterprise data access.
'Copilot trains on your data to improve' – Wrong. Microsoft states that prompts and responses are not used to train the base LLM. This is a key responsible AI principle.
'Copilot can access all data in the tenant' – Wrong. Copilot respects user permissions; it only accesses data the user can already see.
'Copilot is only available in Microsoft 365' – Wrong. There are multiple copilots: GitHub Copilot, Azure AI Copilot, and Windows Copilot.
Specific Numbers and Terms on the Exam
Grounding: The term appears in multiple-choice options. Know that grounding means adding context to the prompt.
Microsoft Graph: The data source for Copilot for Microsoft 365.
Content filters: Know that they block harmful content.
Responsible AI principles: The six principles (fairness, reliability, privacy, etc.) are tested.
Licensing: Copilot for Microsoft 365 requires a separate subscription (e.g., $30/user/month).
Edge Cases and Exceptions
Copilot in Azure: Azure AI Copilot helps manage Azure resources (e.g., 'Create a VM'). It uses Azure CLI and PowerShell behind the scenes.
Copilot in Windows: Windows Copilot provides system-level assistance (e.g., change settings). It is different from M365 Copilot.
Data residency: Copilot processes data in the tenant's region. For multi-region tenants, data may be processed in the home region.
No internet required: Copilot for Microsoft 365 requires internet connectivity; it does not run offline.
How to Eliminate Wrong Answers
If an answer says 'uses your data for training,' eliminate it – Copilot does not.
If an answer says 'works offline,' eliminate it – Copilot requires cloud connectivity.
If an answer says 'accesses all data,' eliminate it – Copilot respects permissions.
If an answer says 'same as ChatGPT,' eliminate it – Copilot includes grounding and filters.
Microsoft Copilot is a generative AI assistant that combines LLMs with your organization's data.
Copilot grounds responses by retrieving data from Microsoft Graph or Azure AI Search.
Prompts and responses are not used to train the underlying model.
Copilot respects user permissions; it only accesses data the user can see.
Content filters block harmful content on input and output.
There are multiple Copilot products: M365, GitHub, Azure, Windows.
Copilot for Microsoft 365 requires a separate license.
Audit logging records all Copilot interactions for compliance.
Copilot can hallucinate; always verify critical information.
Responsible AI principles include fairness, reliability, privacy, etc.
These come up on the exam all the time. Here's how to tell them apart.
Microsoft Copilot
Integrated into Microsoft 365 and Azure.
Grounds responses in user's data via Microsoft Graph.
Includes built-in content filters and responsible AI.
Requires separate subscription (e.g., $30/user/month).
Limited to Microsoft ecosystem.
Azure OpenAI Service
Provides direct access to OpenAI models (GPT-4, etc.).
No automatic grounding; you must provide context manually.
You control content filtering and safety settings.
Pay-as-you-go pricing based on tokens.
Can be used in any application via API.
Mistake
Microsoft Copilot is the same as ChatGPT.
Correct
Copilot is built on similar LLM technology but is integrated with Microsoft 365 and Azure. It grounds responses in your organization's data and applies enterprise-grade security and compliance. ChatGPT is a general-purpose chatbot without enterprise data access.
Mistake
Copilot uses your prompts to train its models.
Correct
Microsoft explicitly states that prompts and responses are not used to train the underlying LLM. This is part of their responsible AI and privacy commitments.
Mistake
Copilot can access any data in your organization.
Correct
Copilot only accesses data that the user has permission to view. It respects existing security boundaries and does not bypass access controls.
Mistake
Copilot is only available in Microsoft 365.
Correct
There are multiple Copilot products: Copilot for Microsoft 365, GitHub Copilot, Azure AI Copilot, and Windows Copilot. Each serves different use cases.
Mistake
Copilot always provides accurate information.
Correct
Copilot can still produce hallucinations or incorrect information, especially if grounding data is insufficient. Users should verify critical information.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
Microsoft Copilot is an enterprise AI assistant that integrates with Microsoft 365 and Azure. It grounds responses in your organization's data (emails, files, calendar) and applies content filters and responsible AI principles. ChatGPT is a general-purpose chatbot that uses OpenAI models without enterprise data integration. Copilot does not use your data for training, while ChatGPT's free tier may use conversations for improvement (with opt-out).
Copilot processes your prompts and responses in real-time. Microsoft states that these are not used to train the base LLM. However, interactions are logged in the Microsoft 365 audit log for compliance purposes. The data you provide (e.g., documents retrieved) remains within your tenant's boundary. Microsoft does not retain your prompts beyond the session.
No, Copilot requires an internet connection to access the cloud-based LLM and your data from Microsoft Graph. There is no offline mode. This is because the model inference happens in Azure data centers, and the grounding data is fetched from cloud services.
Copilot uses the same permissions as Microsoft 365. When it retrieves data from Microsoft Graph, it checks the user's access rights. If a user cannot view a file in SharePoint, Copilot cannot include it in the response. This is enforced by the underlying security model.
Copilot for Microsoft 365 costs $30 per user per month (as of 2025). GitHub Copilot has individual ($10/month) and business ($19/month) plans. Azure AI Copilot is included with Azure subscriptions but may have usage costs. Windows Copilot is free with Windows 11. Always check the latest pricing on Microsoft's site.
Currently, Copilot for Microsoft 365 is primarily text-based. However, it can integrate with other AI services like Microsoft Designer (formerly Bing Image Creator) to generate images if a plugin is used. The AI-900 exam focuses on text generation and data grounding.
Grounding is the process of adding relevant data to the prompt before sending it to the LLM. For example, if you ask 'Summarize my last email,' Copilot retrieves the email from Exchange and appends it to the prompt. This ensures the model has context and reduces hallucinations.
You've just covered Microsoft Copilot and AI-900 — now see how well it sticks with free AI-900 practice questions. Full explanations included, no account needed.
Done with this chapter?