Without understanding agentic AI, you might think that AI can only respond to a single command in isolation, like a search engine that gives you one answer and then forgets you exist. This chapter explains agentic AI, which is the concept of an AI system that can take a complex goal, break it down, plan a sequence of actions, use tools to execute those actions, and then reflect on the results to improve its own performance. For the AI-102 exam, this is a critical topic because Microsoft is heavily investing in creating platforms like Azure AI Agent Service and Copilot Studio that let developers build these autonomous systems, and you need to know how they work and how to plan them.
Jump to a section
A simple way to picture Agentic AI and Agent-Based Solutions
A head chef in a busy professional kitchen is the right image here. This isn't just any cook; this is the chef who plans the entire service. They receive the ticket (the user's request) and immediately begin breaking it down. The ticket says 'Table 7 needs a vegan starter, a medium-rare steak, and a gluten-free dessert.' The chef doesn't personally chop every vegetable or sear every steak. Instead, they orchestrate a team of specialists. They assign the vegetable prep to the commis chef, the steak to the grill chef, and the dessert to the pastry chef. Each of these station chefs is like a specialised tool or a function in your code. The head chef monitors the progress, checking that the grill chef isn't burning the steak and that the pastry chef has enough gluten-free flour. If the grill chef gets behind, the head chef might reassign the vegetable prep to a different station to keep everything running on time. The chef makes real-time decisions based on changing conditions, not just following a fixed recipe. This is directly analogous to an AI agent: the head chef perceives the environment (the kitchen chaos), plans the actions (assigning tasks), executes the orders (cooking the food), and reflects on the results to improve for the next service. A simple script would be like a single cook who can only follow one recipe at a time, unable to handle a busy, dynamic restaurant. The head chef is the intelligent agent, coordinating multiple tools and adapting to achieve a complex goal: serving a perfect meal within a set time.
Let's start with the basics. An 'agent' in the context of AI is a software entity that can perceive its environment, make decisions, and take actions to achieve a specific goal. Think of it as a digital assistant that doesn't just answer questions but does things. 'Agentic AI' is the broader field of building these agents, and an 'agent-based solution' is a system that uses one or more of these agents to solve a problem.
The key difference between a traditional AI solution and an agentic one is autonomy. Traditional AI is often 'stateless'. It takes one input, processes it, and gives one output. A chatbot that answers 'What is the weather?' is stateless. Agentic AI is 'stateful'. It remembers the conversation, it has a goal, and it can take a series of steps to achieve that goal. For example, an agentic travel agent AI might first ask you about your budget, then search for flights, then book the flight, then send you a confirmation email. Each step uses a different tool or function, and the agent decides which tool to use next based on its current understanding of the goal.
How does an agent actually work? It usually follows a cognitive architecture often simplified into a cycle: Perceive, Plan, Act, and Reflect.
First, the agent 'perceives' its environment. This means it receives an input, like a user's message in natural language, or a change in a database, or a sensor reading. This input is converted into a format the agent can understand, usually involving a large language model (LLM) to parse the user's intent.
Second, the agent 'plans'. This is the most complex step. The agent breaks the user's goal into a series of smaller sub-goals or steps. This is often done using a technique called 'chain-of-thought' reasoning, where the LLM thinks step-by-step about what needs to happen. For the travel agent example, the plan might be: 1. Determine departure city. 2. Determine destination. 3. Determine travel dates. 4. Search for flights using the flight API. 5. Present options to user. 6. Book the flight.
Third, the agent 'acts'. It executes each step in the plan. This is where 'tool use' comes in. The agent doesn't know how to call APIs directly; instead, it is given a set of 'tools' or 'functions' it can invoke. These tools are defined in a way that the LLM can understand, like a JSON schema describing 'searchFlights(departure, destination, date)'. The agent decides which tool to call and with what parameters, based on its current plan.
Fourth, the agent 'reflects'. After an action is completed, the agent observes the result. Did the API call succeed? Did it return the expected data? If not, the agent might re-plan, choosing a different tool or asking the user for clarification. This reflection loop is what makes the agent 'adaptive' and different from a simple script.
There are several types of agents you should know for the exam:
Single-agent systems: One agent handles the entire task. Simple but limited in scope.
Multi-agent systems: Multiple agents collaborate. One agent might be a 'planner', while another is a 'worker' or 'specialist' in a particular domain. This is like the head chef and their team of station chefs.
Reactive agents: These respond directly to input without complex planning. Like a reflex. Think of a simple thermostat.
Deliberative agents: These maintain an internal model of the world and reason about their actions before acting. The travel agent is deliberative.
Why does this matter for Azure? Microsoft provides several services to build these agents. Azure AI Agent Service is a managed service that handles the orchestration of these agents, the state management, and the tool integration. It integrates with Azure OpenAI to use powerful LLMs as the 'brain' of the agent. Copilot Studio is another service designed for non-developers to create conversational agents that can also be made agentic by adding plugins (tools) and actions.
What does this replace? Before agentic solutions, if you wanted to automate a complex workflow, you had to write rigid code that followed a fixed flowchart. If a step failed, the whole process broke. You had to manually handle every possible edge case. Agentic AI replaces this with a dynamic, adaptable system that can handle unexpected input and change its approach on the fly. It moves from 'code that controls the AI' to 'AI that controls the code'.
Define the Goal
The first step in planning an agentic solution is to clearly articulate the high-level objective the agent must achieve. This is not a single instruction but an overarching goal, like 'Process customer returns'. The goal is used in the system message to guide the LLM's planning. A poorly defined goal leads to unpredictable agent behaviour.
Identify the Tools
An agent cannot act without tools. In this step, you list every external system or data source the agent will need to interact with. For a returns agent, tools might include 'lookupOrderID', 'generateReturnLabel', 'refundPayment', and 'sendEmail'. Each tool is described with a JSON function schema that the LLM can understand and call.
Design the System Message
This is the prompt that defines the agent. You write instructions that set the agent's role, rules, and step-by-step guidance. For example: 'You are a friendly returns assistant. Always start by verifying the order number using the lookupOrderID tool. Never issue a refund without first checking the order status.' This message is the core of the agent's behaviour.
Implement State Management
You must configure how the agent remembers the conversation. This is often done by storing the conversation history in a database or using a built-in state store in Azure AI Agent Service. The state includes previous user messages, the agent's actions, and the current status of the goal. Without state, the agent would forget what it has already done.
Test and Iterate
You deploy the agent in a controlled environment and run test scenarios. You monitor the agent's decisions through logs. If the agent calls the wrong tool or follows a bad plan, you do not fix code; you refine the system message, improve the tool descriptions, or add new reflection logic. This iterative prompt engineering is the primary development method for agents.
Deploy with Guardrails
Before going live, you implement safety measures. This includes content filters to block inappropriate inputs or outputs, rate limiting to prevent abuse, and 'human-in-the-loop' checks for high-stakes actions like processing a refund. You also set up monitoring alerts to detect unusual agent behaviour.
Let's walk through a concrete scenario. You work for a large insurance company called 'SecureLife Insurance'. The company gets hundreds of emails a day from customers wanting to file claims. Currently, a human agent reads each email, checks the policy details in a database, and replies with a form for the customer to fill out. This is slow and expensive. Your boss asks you to build an AI solution to automate this process.
As an IT professional, you wouldn't just build a simple chatbot. You would design an agentic solution. Here is the step-by-step process you would follow:
First, you would identify the 'goal' of the agent: 'Process a new insurance claim from an email and initiate the claim in our system.'
Second, you would define the 'triggers'. This is the 'perceive' step. A new email arriving in the support mailbox triggers the agent. You set up an Azure Logic App or a Function App to listen for new emails and send the email content to the agent.
Third, you would define the 'tools' the agent can use. You would create functions that the agent can invoke, such as: - 'lookupPolicy(policyNumber)': Takes a policy number from the email and queries your Azure SQL database to get the customer's details and coverage. - 'sendEmailReply(emailAddress, messageBody)': Sends a predefined or dynamically generated email back to the customer. - 'createClaimRecord(customerId, policyId, accidentDescription, date)': Creates a new record in your Azure SQL database. - 'escalateToHumanAgent(customerId, reason)': If the agent cannot understand the email, it sends a notification to a human claims handler in Microsoft Teams.
Fourth, you would configure the 'planning' logic. Using Azure AI Agent Service, you would create an agent with a system message. The system message is a set of instructions that tells the LLM how to behave. It would say: 'You are an insurance claims assistant. Your goal is to process new claims from emails. You must first extract the policy number from the email. If you find it, use the lookupPolicy tool. If the policy is valid, ask the customer for the date and description of the incident. Once you have that, use the createClaimRecord tool. If you cannot find a policy number, use the escalateToHumanAgent tool.' This prompt guides the agent's planning and tool selection.
Fifth, you would test and monitor the agent. You run test emails through it. You would use Azure Monitor to log every step the agent took, every tool it called, and every decision it made. This is crucial for debugging and for compliance in an insurance context. You need to be able to prove that the agent made the right decision.
Finally, you would go live. The agent now handles 80% of incoming claims emails without human intervention. The human agents only deal with the complex cases. The agent also continuously improves its its performance by logging its reflections. If it sees that it often mis-extracts a policy number from a poorly formatted email, you can update the system message or add a better parsing tool. This is the core job of the AI engineer: designing, building, deploying, and maintaining these dynamic, tool-using agents.
The AI-102 exam objective 6.1 'Plan and implement agentic AI solutions' is one of the trickier areas because it is relatively new and fast-moving. The exam will test your conceptual understanding, not your ability to write code. You need to memorise the key Azure services and the core concepts.
Here are the specific topics and question patterns you will encounter:
Service identification: The exam loves to ask you to name the right Azure service for a given scenario. You must be able to distinguish between:
Azure AI Agent Service (for developers, code-heavy, full control).
Copilot Studio (for business users, low-code, good for simple conversational agents with plugins).
Azure AI Foundry (the portal where you build and manage AI projects, including agents).
Azure Functions (this is NOT an agent service, it's a serverless compute service. A common trap is to suggest Azure Functions for an agent task when you should use Azure AI Agent Service).
Tool definition: You will be asked about how to define tools for an agent. The correct format is a JSON schema that describes the function, its parameters, and their types. You must know that the agent uses these schemas to decide which tool to call. A trap is suggesting that you just give the agent a list of API endpoints; the correct answer is that you provide a function definition schema.
The LLM as the planner: The exam will test your understanding that the large language model (LLM) is the 'brain' that does the planning. It is not the agent itself. The agent is the orchestration layer that manages state, tools, and the LLM. A common trap question: 'What component is responsible for breaking down a user request into steps?' Answer: The LLM, guided by the system message.
State management: You need to understand that agentic AI requires state. The agent must remember the conversation history and the current progress toward the goal. The exam might present a scenario where a simple stateless bot fails, and you need to propose a stateful agent solution. The correct answer will involve using a conversation history or memory store.
Multi-agent vs. single-agent: The exam will test when to use a single agent versus multiple agents. A multi-agent system is appropriate when tasks are complex and distinct. For example, one agent for customer authentication, another for product recommendation, and a third for order processing. A trap is recommending a multi-agent system for a simple task that a single agent can handle, which adds unnecessary complexity.
Reflection and iteration: The exam will likely ask about how an agent handles failure. The correct concept is 'reflection loop' or 'iterative planning'. You must know that the agent checks the result of a tool call and if it fails, it can re-plan. A trap is suggesting that the agent simply stops and asks the user for help every time; the correct answer is that the agent should first try to recover automatically using a different approach.
Security and Responsible AI: The exam will test that you must implement guardrails for agents. Agents can have unintended consequences if they take actions autonomously. You need to be familiar with concepts like 'content filtering' on inputs and outputs, 'human-in-the-loop' for critical actions (like sending an email or making a purchase), and 'rate limiting' to prevent abuse.
The 'Planner' role: Some exam questions may mention a 'Planner' component. This is a concept from the Semantic Kernel SDK, another way to build agents. You don't need deep knowledge of Semantic Kernel, but you should know that a 'Planner' is a component that, given a goal and a set of functions, generates a plan for the agent to execute. This is in contrast to the 'LLM-native planning' approach in Azure AI Agent Service.
Memorise the cycle: The Perceive-Plan-Act-Reflect cycle is a framework the exam loves. Be ready to identify which step of the cycle a given action corresponds to. For example, 'The agent calls an API to get the weather' is the 'Act' step. 'The agent analyses the API response and decides it needs to call a different API' is the 'Reflect' and re-plan step.
Agentic AI solutions use a perceive-plan-act-reflect cycle, making them autonomous and adaptable rather than stateless and reactive.
An agent uses tools (defined by function schemas) to interact with the external world; the LLM decides which tool to call based on the current goal and context.
The system message (or prompt) is the most important piece of an agent's configuration, as it defines the agent's persona, goals, available tools, and behavioural boundaries.
Azure AI Agent Service is the primary service for developers to build custom, code-centric agentic solutions, while Copilot Studio targets low-code creators.
State management is essential for agents to remember context and progress across multiple turns of conversation, differentiating them from simple stateless APIs.
A multi-agent system is useful when tasks are diverse and require different specialisations, but it increases complexity and coordination overhead.
Monitoring and logging every agent action is critical for debugging, compliance, and continuous improvement in production systems.
Human-in-the-loop design patterns should be used for high-risk actions, not for every decision, to balance safety with efficiency.
These come up on the exam all the time. Here's how to tell them apart.
Agentic AI Solution
Goal-oriented and plans a sequence of steps to achieve a complex objective.
Autonomous and dynamic; can handle unexpected input by re-planning.
Uses an LLM as a reasoning engine to decide which tool to call next.
Traditional Script/API
Executes a fixed, pre-determined sequence of commands.
Stateless and brittle; if an unexpected input occurs, it breaks or needs explicit error handling.
Does not use an LLM; the logic is hard-coded in if-else statements.
Azure AI Agent Service
Code-first; requires writing JSON schemas and using SDKs like Python or C#.
Designed for developers who need full control over the agent's architecture and tools.
Supports multi-agent patterns and custom state management.
Copilot Studio
Low-code/no-code; built with a graphical interface for creating conversational flows.
Designed for business users and citizen developers to build simple agents.
Best for scenarios with pre-built connectors and less complex orchestration.
Single-Agent System
One agent handles all tasks from start to finish.
Simpler to design, debug, and maintain.
Can become a bottleneck if tasks are highly specialised or the agent's context grows large.
Multi-Agent System
Multiple agents, each specialised for a specific role, collaborate.
More complex to coordinate, requiring a communication protocol between agents.
Better suited for complex workflows where different expertise is needed (e.g., a planner agent, a researcher agent, a writer agent).
Stateless AI (Simple Chatbot)
Does not remember past interactions; each request is independent.
Cannot maintain progress toward a multi-step goal.
Simpler to build but limited in functionality for complex tasks.
Stateful AI (Agent)
Maintains conversation history and progress toward the goal across multiple turns.
Can ask clarifying questions and remember the context of the entire task.
Requires a state store (like a database or in-memory cache) to manage the conversation history.
Human-in-the-Loop (HITL)
Agent requires human approval before executing high-risk actions.
Provides safety and accountability for sensitive operations.
Slower than autonomous systems because of the human review step.
Fully Autonomous Agent
Agent executes all steps without human intervention.
Faster and more efficient for low-risk, well-defined tasks.
Requires robust monitoring and guardrails to prevent unintended consequences.
Mistake
An AI agent is just a chatbot with a fancy name.
Correct
An AI agent is fundamentally different from a simple chatbot. A chatbot is typically stateless and reactive, only responding to the last message. An agent is stateful, goal-oriented, and actively takes actions using tools to achieve a complex objective. It can plan, execute, and reflect on its own actions.
This mistake is common because the user interface for both can look the same: a chat window. People see a conversation and assume it's just a chatbot. The underlying architecture of planning and tool use is invisible to the end user.
Mistake
All AI agents are built the same way and use the same underlying technology.
Correct
There are different types of agents (reactive, deliberative, single-agent, multi-agent) and different architectures for building them. On Azure, you can build agents using Azure AI Agent Service, Copilot Studio, or even custom code with OpenAI functions and Semantic Kernel. The right technology depends on the complexity of the task, the need for customisation, and the developer's skill level.
This misconception arises from marketing that often presents a one-size-fits-all solution. Beginners see 'AI Agent' as a single product rather than a design pattern that can be implemented in many ways.
Mistake
An agent doesn't need any special instructions; the LLM just figures it out.
Correct
An LLM is the brain, but it needs a 'system message' or 'prompt' that defines the agent's role, its goals, the tools it has, and how it should behave. Without this structure, the LLM can hallucinate, call the wrong tool, or ignore safety constraints. Careful prompt engineering is the most critical part of building a reliable agent.
People often overestimate the capabilities of raw LLMs. They see impressive demos and assume the LLM can handle any task autonomously. The reality is that an LLM without guidance is like a brilliant but disorganised intern who needs a very clear brief.
Mistake
If an agent makes a mistake, you just fix the code.
Correct
Agents are dynamic systems. Their behaviour is determined by the combination of the LLM, the system prompt, the tools, and the data. A mistake might not be a bug in code but a flaw in the prompt, a poorly designed tool, or an unexpected edge case in the data. Fixing it often requires iterative prompt engineering, adding better validation in the tools, or improving the reflection logic.
This mistake stems from traditional software development thinking, where bugs are isolated in specific lines of code. Agents are probabilistic systems, so the same code can produce different responses. Debugging an agent requires a different mindset focused on patterns of behaviour rather than specific logical errors.
Mistake
You always need a human to approve every action an agent takes.
Correct
Agents can operate in fully autonomous mode for low-risk tasks, like looking up information from a database. You only need 'human-in-the-loop' for high-risk actions like sending an invoice, deleting a record, or posting to social media. The level of autonomy should be designed based on a risk assessment of each action the agent can perform.
People are rightly concerned about AI running wild. This leads to the assumption that all autonomy is dangerous. In practice, an agent that needs human approval for every little step (like checking the date) would be useless and slower than a human. The goal is to automate low-risk decisions and escalate high-stakes ones.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
A regular function or API call is deterministic and takes a single input to give a single output. An AI agent is autonomous and goal-oriented; it decides which function to call, in what order, and can change its plan based on results. A function call is a tool an agent uses, not the agent itself.
It depends on the service. Azure AI Agent Service is code-heavy and requires understanding of APIs and JSON schemas. Copilot Studio is low-code and designed for business users. For the AI-102 exam, you only need to understand the concepts, not write code.
A well-designed agent has a 'reflection' step where it evaluates its own actions. If it fails to call a tool or gets an error, it can re-plan. If it continuously fails, it should be designed to escalate to a human operator. Logging and monitoring are crucial to catch these failures.
Not automatically in a production sense. The agent itself does not update its own prompt or tools. However, you can analyse logs of its actions and use that data to manually refine the system message, improve tool descriptions, or add new tools. This is an iterative development process, not self-learning.
A 'Planner' is a component, often from the Semantic Kernel SDK, that automatically generates a sequence of steps (a plan) for an agent to follow to achieve a goal. It is an alternative to having the LLM plan on its own. The exam tests that a Planner can generate an order of operations, not the LLM directly.
They are closely related but not identical. Agentic AI refers to the design pattern of building agents that plan and act. Autonomous AI is a broader term that implies an AI system can operate without any human intervention at all. An agentic solution can have degrees of autonomy, from fully constrained to highly autonomous.
You've finished Agentic AI and Agent-Based Solutions. Continue through the AI-102 study guide to build a complete picture of the exam.
Done with this chapter?