What Is Azure Pipelines in DevOps?
On This Page
What do you want to do?
Quick Definition
Azure Pipelines is a tool that helps software teams automatically build and test their code every time they make changes. It then deploys the finished software to wherever it needs to go, like a server or a cloud service. This saves time and reduces errors by doing these repetitive tasks automatically.
Common Commands & Configuration
trigger:
branches:
include:
- main
- releases/*
exclude:
- releases/old/*YAML pipeline trigger configuration that runs the pipeline when changes are pushed to the main branch or any branch under releases/, but excludes releases/old/ branches.
Exams test understanding of trigger syntax, especially how CI triggers differ from PR triggers. Expect scenarios where you must include/exclude specific branches or paths.
steps:
- script: echo 'Hello, world!'
displayName: 'Run a one-line script'Simplest step that runs a script (bash/cmd) inline. Useful for quick tasks like printing variables or testing agent capabilities.
Appears in exams to contrast inline scripts with script file tasks (e.g., 'script' vs 'bash' vs 'powershell'). Tests ability to choose the correct task for cross-platform builds.
pool:
vmImage: 'ubuntu-latest'Specifies the Microsoft-hosted agent pool with an Ubuntu image. Commonly used for Linux builds or cross-platform validation.
Exams ask about hosted vs self-hosted agents, and which vmImage values are available (windows-latest, ubuntu-latest, macos-latest). Also tests cost implications.
- task: PublishBuildArtifacts@1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'Publishes build artifacts (e.g., compiled binaries) to Azure Pipelines, making them available for release pipelines or manual download.
Frequently tested: difference between 'PublishBuildArtifacts' and 'PublishPipelineArtifact' (both exist, but the latter is newer). Also used in multi-stage YAML questions.
variables:
buildConfiguration: 'Release'
majorVersion: 1
minorVersion: 0Defines custom pipeline variables at the top of a YAML file. Can be used in steps with $(variableName) syntax.
Exams test variable scoping (pipeline, stage, job), overriding variables at queue time, and how to use variables in expressions (e.g., ${{ variables.buildConfiguration }}).
- task: DotNetCoreCLI@2
inputs:
command: 'test'
projects: '**/*Tests/*.csproj'
arguments: '--configuration $(buildConfiguration)'Runs .NET Core tests using the DotNetCoreCLI task. Filters test projects with a wildcard pattern and passes configuration from a pipeline variable.
Common in AZ-400 exams: 'DotNetCoreCLI@2' also supports 'build', 'publish', 'restore'. Tests know when to use this vs the older 'VSBuild' task.
resources:
repositories:
- repository: templates
type: git
name: MyProject/MyRepo
ref: refs/tags/v1.0Declares a repository resource (another Azure Repos Git repo) that can be used for template includes. Uses a specific tag 'v1.0'.
Exams test resource types (pipelines, repositories, containers, webhooks). Also tests how to reference templates from other repos with '@' syntax.
Azure Pipelines appears directly in 187exam-style practice questions in Courseiva's question bank — one of the most-tested concepts on AZ-104. Practise them →
Must Know for Exams
Azure Pipelines appears across multiple Microsoft certification exams, but it is most heavily weighted in the AZ-400: Designing and Implementing Microsoft DevOps Solutions exam. This exam expects you to know how to design a CI/CD strategy, configure build and release pipelines, manage agents and agent pools, integrate with GitHub and Azure Repos, and implement security with service connections and Key Vault. You will see scenario-based questions where you need to choose the correct trigger type, determine whether to use a self-hosted or Microsoft-hosted agent, or select the appropriate task to deploy a container to Azure Kubernetes Service.
In the AZ-104: Azure Administrator exam, Azure Pipelines appears in the context of 'implement and manage storage' and 'deploy and manage compute resources.' You may be asked to identify the correct tool for automating deployments, or to troubleshoot a failed deployment by examining pipeline logs. While not as deep as AZ-400, you should understand the basic concepts of CI/CD, build artifacts, and release gates.
For the AZ-900: Microsoft Azure Fundamentals exam, Azure Pipelines is part of the Azure DevOps services. You will only need to recognize it as a CI/CD solution and understand its purpose: to automate building, testing, and deploying code. You may see a question that asks which Azure service should be used to automatically build and deploy applications, and Azure Pipelines is the correct answer.
The exam often presents questions about multistage YAML pipelines versus classic release pipelines. You need to know that YAML pipelines are the modern, recommended approach because they can be version-controlled and reused. Another common question covers agent pools: when you need a specific version of a tool or a GPU, you should use a self-hosted agent. For simple builds, Microsoft-hosted agents are sufficient.
Question types include multiple-choice, multiple-select, and case studies. A typical case study might describe a company that wants to implement a CI/CD pipeline for a .NET Core application hosted on GitHub. You would need to decide on the trigger, the build agent, the artifacts to publish, and the deployment environment. You might also need to set up approval gates for production deployments.
For the AWS certification exams listed (AWS Cloud Practitioner, AWS Developer Associate, AWS Solutions Architect Associate), Azure Pipelines is not directly tested but could appear as a comparison point. You should understand that it is a competitor to AWS CodePipeline, and the core concepts of CI/CD are transferable. The exam objectives for these AWS exams focus on their native services, but knowing the general pipeline model helps.
In Google Cloud exams (Google Associate Cloud Engineer, Google Cloud Digital Leader), the analogous service is Cloud Build. Again, the concepts are similar but Azure Pipelines is Microsoft-specific. You won't need to know Azure Pipelines details for Google exams, but understanding CI/CD patterns is always helpful.
Simple Meaning
Azure Pipelines is like an automated assembly line for software. Imagine you own a factory that makes custom bicycles. Normally, when you get a new order, you must handpick each part, weld the frame, attach the wheels, test the brakes, and then package the bike. If you do this manually every single time, you might make mistakes, forget a step, or take too long. Now imagine you set up a conveyor belt system where each station does one job automatically. A robot picks the frame, another attaches the wheels, a machine tightens every bolt, a tester checks the brakes, and a packer wraps the final product. Once you set up this system, every new order goes through the same process without you having to watch over it.
Azure Pipelines does exactly that, but for software code. Every time a developer writes new code or makes changes, Azure Pipelines automatically runs the entire process: it fetches the code from a repository like GitHub, compiles it, runs tests to make sure nothing is broken, and then sends the finished software to a server or a cloud platform where users can access it. The key idea is "continuous integration" and "continuous delivery," often shortened to CI/CD.
Continuous integration means that every change to the code is automatically integrated and tested. Think of it like a team of chefs cooking a huge pot of soup. If each chef adds their ingredient without checking, the soup might become salty or sour. Instead, they agree to add ingredients one at a time, taste the soup after each addition, and only move forward if it tastes good. That is how CI works: every small change is tested immediately to catch problems early.
Continuous delivery takes it further: after the code passes all tests, it is automatically packaged and ready to be deployed to production. Using the chef analogy, once the soup tastes perfect, a machine automatically ladles it into bowls and serves it to customers without any chef needing to decide when to serve. In software, this means that the latest working version of the application is always ready to be released.
Azure Pipelines supports both Windows and Linux, works with many programming languages like Python, C#, Java, and JavaScript, and can deploy to Azure, AWS, Google Cloud, or even your own servers. It is a central part of Microsoft's Azure DevOps suite, which also includes boards for tracking work, repos for storing code, and test plans for managing testing.
Full Technical Definition
Azure Pipelines is a cloud-native CI/CD service within the Azure DevOps ecosystem. It allows teams to automate the build, test, and deployment phases of software development. Under the hood, Azure Pipelines relies on a set of interconnected components: agents, pipelines (composed of stages, jobs, and steps), triggers, and integration points with version control systems and artifact repositories.
Agents are the compute resources that run the actual build and deployment tasks. There are two types: Microsoft-hosted agents and self-hosted agents. Microsoft-hosted agents are managed by Azure and come pre-configured with common tools for Windows, Linux, and macOS development. They are ideal for teams that do not want to maintain infrastructure. Self-hosted agents run on your own machines, giving you full control over the environment, which is necessary for compliance or specialized hardware requirements. The agent communicates with Azure Pipelines using a REST API and a polling mechanism. Each agent belongs to an agent pool, and jobs are assigned to agents from that pool based on capabilities like installed software.
Pipelines are defined using either the classic editor (a visual drag-and-drop interface) or YAML (a code-based definition file). YAML pipelines are preferred because they are version-controlled, repeatable, and easier to maintain. A pipeline consists of one or more stages. Stages represent major divisions in the pipeline, such as Build, Test, and Deploy. Each stage contains one or more jobs, which are units of work that run sequentially or in parallel. Jobs are further broken down into steps, which are the individual tasks like running a script, compiling code, or copying files.
Triggers dictate when a pipeline should start. The most common trigger is a commit to a specific branch in a version control system such as GitHub, Azure Repos, or Bitbucket. You can also set up scheduled triggers, pull request validation triggers, or pipeline completion triggers (where one pipeline starts another). Variables and variable groups allow you to store and manage sensitive information like passwords, API keys, and connection strings. Variable groups can be linked to Azure Key Vault for additional security.
Azure Pipelines uses a concept called "environments" to manage target deployment destinations. An environment can be a Kubernetes cluster, a virtual machine, a web app, or even a specific resource group. Approvals and checks can be added to environments to enforce governance: for example, a manager must approve a release before it goes to production, or a security scan must run first.
Integration with other tools is extensive. Azure Pipelines supports over 200 built-in tasks, including NuGet restore, MSBuild, Docker build and push, Kubernetes manifest deployment, and Azure Web App deploy. Custom tasks can be created using PowerShell, Bash, or any scripting language. The service also integrates with third-party tools like SonarQube for code quality, Jira for issue tracking, and Slack for notifications.
Security is handled through service connections, which are secure links between Azure Pipelines and external services like Azure subscriptions, GitHub repositories, or Docker registries. Service connections use OAuth, managed identities, or certificates to authenticate. The pipeline runs in a sandboxed environment, and secrets are masked in logs.
Scalability is a key feature. Azure Pipelines runs on global infrastructure, so it can scale to thousands of parallel jobs. The free tier offers 1,800 minutes per month for private projects and unlimited minutes for public projects. Paid tiers provide additional concurrency and storage.
Version control of pipeline definitions is enforced through YAML files stored alongside your code in the repository. This means that when you make changes to the pipeline, you can review them through pull requests, track history, and roll back if needed. This is part of the "pipeline as code" philosophy, which treats the build and deployment process as part of the software product itself.
Real-Life Example
Think of a large restaurant chain that serves pizza. The chain has dozens of locations, and every location must serve the same quality pizza. The central kitchen develops a new recipe for a Hawaiian pizza with fresh pineapple. In the old days, the head chef would write the recipe on paper, hand it to each location manager, and trust them to follow it exactly. Some managers would use too much cheese, others would undercook the crust, and some would forget the pineapple entirely. The result was inconsistent and unhappy customers.
Now, the chain uses a "Pizza Pipeline" system. The head chef (developer) creates the new recipe code and uploads it to the central system (GitHub repository). The Pizza Pipeline automatically kicks in. First, it checks each ingredient against the approved list: does the pineapple meet freshness standards? Is the cheese low-fat enough? This is the build phase, where the pipeline verifies dependencies. Then, the pipeline sends the recipe to a test kitchen where a robot makes a sample pizza, bakes it at the right temperature, and even takes a picture to confirm it looks right. This is the test phase, integrated testing of the recipe.
Once the test kitchen approves, the pipeline proceeds to the deployment phase. It pushes the exact recipe, cooking temperature, and plating instructions to every restaurant's digital oven system. Each oven now automatically sets the correct temperature and cooks the pizza for the precise time needed. The restaurants no longer need to interpret handwritten notes. The system also monitors the ovens to ensure they are working correctly and sends alerts if a pizza is burning.
Now compare this to software. The developer writes code (recipe) and pushes it to a repository. Azure Pipelines automatically builds the software (checks ingredients), runs unit tests and integration tests (test kitchen), and then deploys the application to servers (restaurant ovens). The customer always gets the same quality, no matter which server they hit. If the deployment fails, Azure Pipelines can automatically roll back to the previous known-good version, just like the oven reverting to the last proven recipe.
This analogy also highlights the concept of parallel execution. Each restaurant can run its own oven independently, just as multiple servers can receive the deployment simultaneously. The pipeline ensures that every location gets the update at the same time, or in a controlled rollout where 10% of locations get it first, then 50%, then 100%.
Why This Term Matters
In modern IT operations, software is never truly 'finished.' Teams release updates weekly, daily, or even multiple times per day. Doing this manually is impossible at scale. Azure Pipelines matters because it enables organizations to deliver software faster and more reliably. Without a CI/CD pipeline, developers must manually compile their code, run tests, copy files to servers, and restart services. Each manual step introduces the risk of human error, such as forgetting to include a file, using the wrong configuration, or deploying to the wrong server.
Azure Pipelines also enforces consistency. Every build runs in the same environment, with the same tools and dependencies. This eliminates the 'it works on my machine' problem where a developer's local setup differs from production. The pipeline catches integration issues early when a developer's changes conflict with another team member's work. This is critical in large teams where dozens of people commit code to the same repository.
From a business perspective, Azure Pipelines reduces time to market. A feature that used to take a week to test and deploy can now be released in hours. It also improves security because automated scans for vulnerabilities can be inserted into the pipeline, ensuring that insecure code never reaches production. Audit trails are automatically maintained, showing exactly who changed what, when, and whether the deployment succeeded. This is essential for compliance with regulations like SOC 2, HIPAA, or PCI-DSS.
For IT professionals, knowing how to set up and maintain Azure Pipelines is a highly marketable skill. It is central to the DevOps role and is tested in the AZ-400 (Azure DevOps Engineer Expert) exam. It also appears in the AZ-104 (Azure Administrator) exam at a conceptual level, and in the AZ-900 (Azure Fundamentals) exam for basic awareness. Even if you never become a DevOps specialist, understanding pipelines helps you work more effectively with developers and operations teams.
How It Appears in Exam Questions
Questions about Azure Pipelines often fall into a few distinct patterns. The first pattern is scenario-based, where the question describes a development team's workflow and asks you to choose the best pipeline configuration. For example, a team is using GitHub, they want to build and test their application on every push to the main branch, and then deploy to a staging environment only if all tests pass. You would need to select the correct trigger (continuous integration trigger for the main branch), configure the stages correctly, and set up a condition for the deployment job that depends on the test job succeeding. These questions test your ability to translate a real-world requirement into a pipeline design.
The second pattern is troubleshooting. The question presents a failed pipeline run, and you must identify the cause from the logs or from the configuration. Common issues include incorrect YAML indentation, a missing service connection, an agent that does not have the required capabilities, or an environment that has not been created. For example, a pipeline that uses a Docker task might fail because the agent does not have Docker installed. In that case, you would recommend switching to a self-hosted agent that has Docker, or using a Microsoft-hosted agent with the Docker prerequisite added.
The third pattern is configuration-based. These questions ask you to fill in a missing parameter or choose the correct syntax for a YAML pipeline. For instance, you might be given a YAML snippet with a blank for the trigger, and you need to choose 'trigger: none' if it is a pipeline that should only run manually, or 'trigger: branches: include: main' for a CI trigger. Another common configuration question involves defining variables: you may need to choose between a variable defined at the pipeline level, a variable group, or a variable stored in Key Vault.
The fourth pattern is comparison. The question might ask you to differentiate between Azure Pipelines and another Azure DevOps service, such as Azure Boards or Azure Repos. Or you might need to choose between a classic release pipeline and a multi-stage YAML pipeline. The correct answer is almost always in favor of YAML due to its flexibility and source-control integration.
Some questions incorporate approval gates and checks. For example, a company requires a manager to approve any deployment to production. The question asks which feature of Azure Pipelines should be used. The answer is 'approvals' on the environment. You might also be asked about required reviewers on a pull request, which is a branch policy, not a pipeline feature.
Security-related questions are also common. You might be asked how to securely store a database connection string used in a pipeline. The correct answer is to use a variable group linked to Azure Key Vault. Another question could ask about the difference between a service connection and a service principal: the service connection is the abstraction that uses the service principal to authenticate to a target resource.
Finally, watch out for questions about agents. A scenario where a build takes a long time because the agent is waiting for resources might lead to a recommendation to increase agent capacity or use self-hosted agents. If the question mentions compliance or the need for specific software, the answer is self-hosted agents. If it mentions cost savings and simplicity, the answer is Microsoft-hosted agents.
Practise Azure Pipelines Questions
Test your understanding with exam-style practice questions.
Example Scenario
A small startup called 'BookBuddy' is building a web application that lets users track books they have read. The team uses GitHub to store their code, which is written in Python with the Django framework. They have three environments: development, staging, and production. Currently, whenever a developer finishes a feature, they manually copy the code to the staging server via FTP, run tests by typing commands, and then hope nothing breaks. This happens about twice a week. Today, the CEO insisted on moving to automated deployments because a recent manual deployment accidentally overwrote the production database.
You are assigned to set up Azure Pipelines. First, you create a new pipeline in Azure DevOps, connecting it to the GitHub repository. You configure a CI trigger so that every push to the 'develop' branch triggers a build. The build stage runs two steps: first, it installs Python dependencies from requirements.txt, and second, it runs all unit tests using pytest. If any test fails, the entire pipeline stops, and the developer receives an email notification. Once the build succeeds, the pipeline proceeds to the staging deployment stage. Here, it runs a script that uses the Azure CLI to deploy the code to a Linux virtual machine running in Azure. You set the condition to only run the staging deployment if the build stage succeeded.
Next, you set up the production deployment stage. This stage only runs when code is pushed to the 'main' branch. But you also add an approval gate: the team lead must manually approve the deployment to production. You configure a post-deployment wait time of 10 minutes to ensure the application is stable, and you add a smoke test task that sends an HTTP request to the production URL and checks that the response code is 200. If the smoke test fails, the pipeline automatically rolls back to the previous version by redeploying the last known good artifact.
After the pipeline is in place, the team notices a dramatic improvement. When a developer submits a pull request, the pipeline runs automatically and shows whether the tests pass directly in the GitHub pull request interface. Integration issues are caught within minutes, not days. The CEO is happy because the production deployment now requires approval, eliminating the risk of accidental overwrites. The startup can now deploy multiple times a day with confidence.
Common Mistakes
Assuming that Microsoft-hosted agents can run any custom software without configuration
Microsoft-hosted agents only come with a predefined set of tools. If your build requires a specific version of a compiler or a rare library, the agent may not have it installed, causing the pipeline to fail.
Check the list of installed software on Microsoft-hosted agents from Microsoft documentation. If your software is not listed, use a self-hosted agent where you can install whatever you need.
Storing secrets directly in the pipeline YAML file
YAML files are often stored in version control, so hardcoding passwords or API keys exposes them to everyone with access to the repository. This is a major security risk.
Use Azure Key Vault to store secrets and link them to the pipeline using variable groups. Then reference the secret by name, and it will be masked in logs.
Setting the trigger to 'none' by accident and then wondering why the pipeline does not run automatically
If you explicitly set 'trigger: none' in your YAML pipeline, it will only run when manually triggered. New commits will not start the pipeline, which defeats the purpose of CI.
If you want continuous integration on every commit, omit the 'trigger: none' line or specify the branches you want to include, such as 'trigger: branches: include: main'.
Running all jobs in parallel without considering dependencies
Some jobs depend on the output of previous jobs. For example, a deployment job needs the build job to finish first. If you run them in parallel, the deployment may try to use an artifact that does not yet exist.
Use the 'dependsOn' keyword in YAML to specify job dependencies. By default, jobs in the same stage run in parallel, so you must explicitly set dependencies if sequential execution is needed.
Ignoring the difference between stage-level and job-level conditions
A condition at the stage level applies to the entire stage. If you want to skip only one job within a stage, you must apply the condition at the job level. Applying it incorrectly can bypass security checks or skip critical tests.
Use the 'condition' property at the appropriate granularity. For example, use 'condition: succeeded()' at the stage level to ensure the entire stage runs only if the previous stage succeeded. For individual jobs, use 'condition: eq(variables['Build.SourceBranch'], 'refs/heads/main')'.
Forgetting to set the correct service connection for a deployment
Azure Pipelines needs permission to deploy to your Azure subscription. If you use the wrong service connection or an expired one, the deployment will fail with an authorization error.
Create a service connection in Azure DevOps under Project Settings > Service Connections. Ensure it uses the correct Azure subscription and has the necessary permissions (e.g., Contributor role). Then reference that service connection name in your pipeline tasks.
Exam Trap — Don't Get Fooled
{"trap":"A question states that a company needs to deploy to an on-premises server that is behind a firewall and cannot be accessed from the internet. The options include: use a Microsoft-hosted agent, set up a self-hosted agent, or use Azure Clouds. The trap is that learners might choose Microsoft-hosted agent because it is easier, but that agent cannot reach the on-premises server due to the firewall."
,"why_learners_choose_it":"Microsoft-hosted agents are the default choice and are simpler to set up. Learners may not realize that the agent must have network connectivity to the deployment target. They might assume that Azure Pipelines can magically bypass firewalls."
,"how_to_avoid_it":"Remember that agents are the machines that execute pipeline tasks. If the deployment target is on-premises and behind a firewall, you must install a self-hosted agent on a machine inside your network. This agent can then communicate with both the Azure Pipelines service (outbound) and the on-premises server (within the network).
The self-hosted agent only needs outbound connectivity to Azure, which is typically allowed through firewalls."
Commonly Confused With
GitHub Actions is a CI/CD service built directly into GitHub, while Azure Pipelines is a service within Azure DevOps. Both can build, test, and deploy code, but GitHub Actions is more tightly integrated with GitHub repositories, whereas Azure Pipelines supports multiple version control systems including GitHub, Azure Repos, and Bitbucket. For a team already using GitHub, GitHub Actions may be simpler, but Azure Pipelines offers deeper integration with other Azure services.
If you start a new project on GitHub and just want automated builds, you could use GitHub Actions. If your team uses Azure DevOps for work item tracking and repos, Azure Pipelines is the natural choice.
The classic editor is a visual interface for defining pipelines, while the YAML editor is code-based. Many learners confuse the two, thinking the classic editor is the only way. The YAML approach is now recommended because it keeps the pipeline definition in source control, making it versionable and reviewable like any other code.
Classic editor is like using a drag-and-drop website builder. YAML is like writing HTML directly. Both build a website, but HTML gives you more control and can be tracked with version control.
Release Pipelines were the older way to manage deployments separate from build pipelines. In the classic model, a build produces artifacts, and a release pipeline deploys them. With multi-stage YAML pipelines, build and deployment are combined into a single pipeline. The term 'Release Pipeline' is now legacy, but you might still encounter it in older resources or in the classic editor.
Release Pipelines are like two separate conveyor belts: one builds the product, the other packages and ships it. Multi-stage YAML is one long conveyor belt that does everything from start to finish.
Azure Boards is a work tracking system with Kanban boards, backlogs, and sprints. It is used to plan and track work items like user stories and bugs. Azure Pipelines is about automation of builds and deployments. The two services integrate: you can link work items to pipeline runs, but they serve entirely different purposes.
Azure Boards is like a whiteboard with sticky notes listing tasks to do. Azure Pipelines is the robot that actually does the tasks when someone finishes writing code.
AWS CodePipeline is Amazon's equivalent of Azure Pipelines. Both are CI/CD services, but CodePipeline is designed to work within the AWS ecosystem, while Azure Pipelines is platform-agnostic and can deploy to AWS, Google Cloud, or on-premises. Azure Pipelines also has a free tier for public projects, which CodePipeline does not.
If your company uses Azure, you will likely choose Azure Pipelines. If it uses AWS, you might choose AWS CodePipeline, but Azure Pipelines can still work if cross-cloud is needed.
Step-by-Step Breakdown
Create an Azure DevOps organization and project
Everything starts in Azure DevOps. You sign in with a Microsoft account and create an organization (a container for projects). Within the organization, you create a project that holds your pipeline definitions, repos, and other settings. This is the home base for your CI/CD operations.
Connect your source code repository
Azure Pipelines supports GitHub, Azure Repos, Bitbucket, and other Git providers. You authorize the connection during pipeline creation. This link allows the pipeline to access your code, detect changes, and trigger builds. For example, you can choose GitHub and grant Azure Pipelines access to your repositories.
Select a pipeline type: YAML or Classic
You decide whether to define your pipeline in a YAML file stored in your repository, or using the classic visual editor. The YAML approach is the modern, recommended method. It puts the pipeline definition under version control, making it easy to review changes, track history, and reuse across branches.
Define the trigger
The trigger tells the pipeline when to run. You can set a CI trigger to run on every push to a specific branch, or a PR trigger to run when a pull request is created. You can also use scheduled triggers or pipeline completion triggers. The trigger is specified in the YAML file using the 'trigger' keyword.
Set up stages, jobs, and steps
A pipeline is organized into stages (e.g., Build, Test, Deploy). Each stage contains one or more jobs. Jobs are sequences of steps. Steps are the actual tasks: running scripts, using built-in tasks like NuGet restore, Docker build, or Azure Web App deploy. Dependencies between stages and jobs are defined with 'dependsOn' and 'condition'.
Configure agents and agent pools
You choose which agent pool will execute the jobs. Microsoft-hosted agents are pre-configured and automatically managed, suitable for standard builds. Self-hosted agents give you full control and are needed for specialized environments or when you need to run tasks on your own network. Agents are registered in pools, and jobs are assigned to available agents.
Add variables and secrets
Variables allow you to pass values like configuration settings into the pipeline. You can define them inline in the YAML, in variable groups, or in Azure Key Vault. Secrets like passwords are stored securely in Key Vault and linked via variable groups. The pipeline can then reference these variables without exposing them in logs.
Set up environments and approvals
Environments represent the deployment targets, like Production, Staging, or Development. You can add approvals, checks, and gates to control deployments. For example, a manager must approve a deployment to Production. Checks might include verifying that a security scan passed or that no high-severity bugs are open.
Run the pipeline and monitor
Once configured, you commit your YAML file to the repository. The pipeline triggers automatically based on your trigger settings. You can watch the progress in the Azure Pipelines portal. Each step shows logs, and you can drill into failures. Notifications can be set up for build failures, successes, or pending approvals.
Review and iterate
After the pipeline runs, you review the artifacts and deployment outputs. You may adjust the configuration based on failures or feedback. Because the pipeline is code, you can create pull requests to propose changes, run tests on those changes, and merge improvements over time.
Practical Mini-Lesson
In practice, setting up Azure Pipelines involves more than just dragging tasks around. You must understand the architecture of your application and the infrastructure it runs on. Let us walk through a realistic scenario: you have a .NET Core web API that connects to a SQL Server database. You want to automate the build, run unit tests, run integration tests against a test database, and deploy to an Azure App Service.
First, you need to decide on the agent. For a .NET Core build, a Microsoft-hosted agent works well because it comes with the .NET SDK pre-installed. However, if your integration tests require a SQL Server instance running on the agent, you might need a self-hosted agent with SQL Server installed, or you could spin up a temporary database container using a Docker task. The Docker task can run an SQL Server image, wait for it to be ready, and then run your integration tests against it. This is a common pattern and is more reliable than relying on an agent's pre-installed software.
Your YAML pipeline will start with a trigger for the main branch. The first stage is 'Build', which includes a job that restores NuGet packages, builds the project, runs unit tests (using dotnet test), and publishes the build artifacts. You should use the 'PublishPipelineArtifact' task to upload the compiled code and configuration files. This artifact will be used by subsequent stages.
The second stage is 'Test_Integration', which depends on the 'Build' stage. Here, you use the 'DownloadPipelineArtifact' task to get the build output. Then you run a Docker task to start a SQL Server container. After the container is ready, you run your integration tests against it. This is a good place to use variables for the database connection string, which you store in a variable group linked to Key Vault. After tests finish, you stop the container.
The third stage is 'Deploy_Staging', which depends on 'Test_Integration'. In this stage, you deploy the build artifact to an Azure App Service named 'myapp-staging'. You use the 'AzureWebApp' task and specify the service connection. After deployment, you run a smoke test task that sends an HTTP GET to the staging URL and checks for a 200 status code. If the smoke test passes, the pipeline proceeds.
The fourth stage is 'Deploy_Production', which depends on 'Deploy_Staging'. This stage has an approval gate: a deployment manager must approve it. The deployment goes to the production App Service 'myapp-prod'. You may also add a manual intervention step so that the pipeline pauses and waits for approval before executing.
What can go wrong? One common issue is that the service connection fails because the service principal does not have permission on the target App Service. You need to ensure the service principal has the 'Web App Contributor' role. Another issue is that the Docker task cannot pull the SQL Server image because the agent lacks internet access, or the Docker engine is not installed. In that case, you might need to switch to a self-hosted agent with Docker configured. Also, if your integration tests require the database to have specific initial data, you must include a setup script that runs after the container starts.
Professionals also need to consider parallel jobs. If you have many developers pushing code, you might run out of parallel job capacity. You can purchase additional parallelism in Azure DevOps, or you can use self-hosted agents to increase your own capacity. Monitoring is crucial: set up alerts for pipeline failures and integrate with Slack or Microsoft Teams so the team knows immediately when a build breaks.
Finally, remember that Azure Pipelines is not just for .NET. It supports Python, Java, Node.js, Go, and many others. The tasks and tools differ, but the pipeline structure remains the same. The key takeaway is to always define your pipeline as code, keep secrets out of the YAML, and use approvals for sensitive deployments.
Troubleshooting Clues
Pipeline stuck in 'queued' status indefinitely
Symptom: Pipeline does not start executing; remains in 'queued' state for hours.
No agent is available to pick up the job. This can happen if the pool has zero agents online (self-hosted agents offline) or all Microsoft-hosted agents are busy (parallel jobs exhausted).
Exam clue: Exams present a scenario where a pipeline won't run, and you must identify if the agent pool is misconfigured or if the organization has run out of parallel job capacity.
YAML syntax error: 'A mapping was found which is not a valid child of a sequence'
Symptom: Pipeline fails to save with a YAML parse error; often indentation-related.
YAML is whitespace-sensitive. A step or job is incorrectly indented, e.g., mixing spaces and tabs, or placing a step at the wrong level (e.g., under 'pool' instead of 'steps').
Exam clue: Exams may show malformed YAML and ask what the error is. They test awareness of YAML indentation rules (2 spaces recommended).
Task fails with 'Access denied' when accessing Azure Key Vault
Symptom: Error: 'Failed to fetch secret' or 'Unauthorized' during a task that reads secrets from Key Vault.
The pipeline's service principal connection lacks the 'Get' and 'List' permissions on the Key Vault's access policy. Or the Azure Resource Manager service connection is not set up with the correct subscription.
Exam clue: Exams test how to grant Key Vault permissions via Access Policies (not IAM) for pipeline tasks. Also tests difference between 'Azure Key Vault' task and referencing secrets in variables.
Pipeline fails with '##[error]The process '/usr/bin/docker' failed with exit code 1'
Symptom: Docker commands fail during a build; exit code 1.
Typically the Docker daemon is not running, the Dockerfile path is wrong, or the Docker image name has invalid characters. Also common when using self-hosted agents without Docker installed.
Exam clue: Exams ask about prerequisites for Docker tasks: ensure Docker is installed on the agent, check the 'Dockerfile' path, and use the 'Docker@2' task with correct 'command' input.
Pipeline artifact not available in release pipeline
Symptom: Release pipeline shows 'No artifacts found' or cannot download the build output.
The build pipeline did not use the 'PublishBuildArtifacts' or 'PublishPipelineArtifact' task, or it published to a different artifact name. Release pipeline's artifact source may point to the wrong build definition.
Exam clue: Exams test the difference between build artifacts and pipeline artifacts, and how to link them in a classic release or multi-stage YAML pipeline.
Self-hosted agent offline or not responding
Symptom: Agent shows as 'offline' in Agent Pools; jobs targeting that pool stay queued.
The agent machine may be powered off, the agent service stopped, or the agent configuration token expired. Also network connectivity issues to Azure DevOps can cause this.
Exam clue: Exams test troubleshooting steps: check agent logs in C:\vsts-agent\_diag (Windows) or ~/vsts-agent/_diag (Linux), and run config.cmd (or config.sh) to re-register if token expired.
'Variable group not found' error when referencing a variable group
Symptom: Error at runtime: 'Variable group 'MyGroup' does not exist'.
The variable group name is misspelled, or the group was deleted. Also, the pipeline must have permission to access the library variable group (security scope).
Exam clue: Exams ask about variable groups vs variables, and how to link variable groups via YAML: 'variables: - group: MyGroup'. Also tests that library items are secure by default and need permissions.
Learn This Topic Fully
This glossary page explains what Azure Pipelines means. For a complete lesson with labs and practice, see the topic guide.
Covered in These Exams
Current Exam Context
Current exam versions that test this topic — use these objectives when studying.
Related Glossary Terms
An approval gate is a checkpoint in an Azure DevOps release pipeline where a pipeline run pauses until a designated team or person manually approves or rejects the deployment.
Azure Artifacts is a service within Azure DevOps that allows teams to create, host, and share packages like NuGet, npm, Maven, and Python, making software dependencies easier to manage across projects.
Azure Boards is a work tracking and project management service from Microsoft that helps teams plan, track, and discuss work across the software development lifecycle using Kanban boards, backlogs, and dashboards.
Azure DevOps is a Microsoft service that provides development tools for planning, building, testing, and deploying software applications using automated pipelines and collaboration features.
Azure Repos is a set of version control tools that allow teams to manage their source code, track changes, and collaborate on software projects using Git or Team Foundation Version Control (TFVC) within the Microsoft Azure ecosystem.
Azure Test Plans is a service within Azure DevOps that provides a rich set of tools for manual and exploratory testing, including test case management, test execution, and stakeholder feedback, all integrated into the CI/CD pipeline.
Quick Knowledge Check
1.You want to trigger an Azure Pipeline only when changes are pushed to the 'main' branch and exclude any changes to markdown files. Which YAML trigger configuration should you use?
2.Your pipeline needs to run on a self-hosted agent with specific software installed. Which pipeline setting ensures the job runs on that agent?
3.A build pipeline publishes artifacts, but the release pipeline says 'No artifacts found'. What is the most likely cause?
4.In a multi-stage YAML pipeline, you want to run a job only if the pipeline was triggered by a PR. Which condition should you use?
5.Your pipeline uses an Azure Key Vault variable group to store a connection string. The pipeline fails with 'Access denied' when accessing the secret. What should you do?