# Implementing a Build Pipeline

> Chapter 6 of the Courseiva AZ-400 curriculum — https://courseiva.com/learn/az-400/implementing-a-build-pipeline

**Official objective:** 3.2 — Implement a build pipeline

## Introduction

How do you ensure that the code you write today will still work perfectly when your team adds new features next month? The answer is a build pipeline, and it's the engine that turns raw code into reliable software, automatically catching problems before they reach your users. For the AZ-400 exam, understanding how to implement a build pipeline is absolutely critical because it's the foundation of every other DevOps practice you'll learn.

## The Catering Kitchen Analogy

Have you ever wondered how a restaurant prepares a complex meal like a three-course banquet for hundreds of guests, without chaos?

The secret is a structured process, much like a build pipeline. Think of the head chef as the developer who writes the recipe (the code). But the recipe alone isn't the meal. The kitchen needs a system to turn that recipe into a finished dish, every single time. That system is the build pipeline.

First, the head chef submits the recipe to the kitchen manager (the build trigger). The manager checks if the ingredients (source code) are available and fresh. Then, they hand the recipe to the prep cooks (the build agent) who follow a precise series of steps: chop vegetables (compile code), measure spices (run unit tests), and prepare the sauce (package dependencies). Each step is automated. The prep cook doesn't decide what to do next; the recipe card (the pipeline YAML file) tells them.

If the prep cook tries to add salt when the recipe says sugar, the system stops and alerts the chef (a failed build). If everything is perfect, the dish is plated and sent to the pass (the build artifact), ready for the waiters (the release pipeline) to serve.

Before this system, chefs would shout instructions across a chaotic kitchen, leading to burnt dishes and inconsistent meals. The build pipeline standardises the process, catches mistakes early, and ensures every meal (software release) is identical to the one before. It's the difference between a frantic home cook and a Michelin-starred kitchen.

## Core explanation

A build pipeline is an automated set of steps that turns source code from a repository (a central storage place for your code, like GitHub or Azure Repos) into a deployable product, often called a build artifact. Before build pipelines, developers would manually compile code, run tests, and package files on their own computers. This was slow, error-prone, and led to the classic 'it works on my machine' problem. A build pipeline solves this by automating everything, creating a repeatable, consistent process.

Here is how it works, step by step.

First, something triggers the pipeline. This is usually a change to the code in the repository. When a developer `git push`s their changes to a branch (like `main` or `feature/xyz`), a webhook sends a signal to Azure DevOps, telling it, 'New code is ready.' This is called a Continuous Integration (CI) trigger. CI means you integrate code changes into the main codebase frequently, and each integration is verified by an automated build.

Second, Azure DevOps allocates a build agent. A build agent is a virtual machine (a computer in the cloud) or a dedicated server that has all the tools installed to build your specific software. For example, if you're building a .NET application, the agent has the .NET SDK installed. If you're building a Node.js app, it has Node.js and npm. Azure DevOps provides Microsoft-hosted agents, which are maintained by Microsoft and have many common tools pre-installed, or you can use self-hosted agents you manage yourself.

Third, the pipeline executes a series of tasks defined in a YAML file (a human-readable configuration file) or through the Azure DevOps classic editor. Each task is a specific action, like 'Run npm install', 'Run unit tests', 'Compile the code', or 'Publish the output'. These tasks run sequentially unless you configure them to run in parallel. The order matters: you must install dependencies before you can compile, and you should run tests before you publish.

Fourth, the pipeline produces an artifact. An artifact is the packaged, compiled output — a .zip file, a Docker container image, a .war file, or a compiled executable. This artifact is the 'thing' that you will later deploy to a server. The pipeline publishes this artifact to a storage location, often called a 'drop' folder or a container registry.

Fifth, the pipeline reports the result. If every task succeeds, you get a green checkmark. If any task fails (a test fails, the code doesn't compile, a security scan finds a vulnerability), the pipeline stops and marks itself as 'Failed'. The developer who pushed the code gets an email or a notification with the error logs, allowing them to fix the problem immediately.

Why is this so important for DevOps? It enforces quality gates. A quality gate is a checkpoint that passes or fails the pipeline. For example, you can set a quality gate that says 'All unit tests must pass' or 'Code coverage must be above 80%'. If the code doesn't meet these criteria, it never becomes an artifact, and it never moves to the deployment stage. This prevents bad code from ever reaching users.

There are a few key concepts to know for the exam:

- CI (Continuous Integration): the practice of automatically building and testing every code change. The build pipeline is the tool that implements CI.
- CI Trigger: what starts the pipeline. Common triggers include 'a push to a branch', 'a pull request being created', or 'a schedule' (nightly builds).
- Build Agent: the machine that runs your pipeline tasks. You choose between Microsoft-hosted (less maintenance, less control) and self-hosted (more control, more maintenance).
- Artifact: the output of a successful build. It's what you deploy.
- YAML: the modern way to define pipelines in Azure DevOps. It is a text file that lives in your repository alongside your code, so your pipeline is versioned.

To summarise: a build pipeline takes code, runs automated checks, and packages it into an artifact. It is the 'automation' in 'automated builds', and it is the first and most critical step in any modern software delivery process.

## Real-world context

Sarah is a junior developer at a small e-commerce company called 'ShopFast'. They sell customisable mugs online. Her boss wants to add a new feature: letting customers upload a photo to print on the mug. Sarah writes the new code on her laptop, and it works fine when she tests it. She pushes her changes to the `main` branch on GitHub.

Behind the scenes, a webhook fires, and the build pipeline in Azure DevOps kicks off automatically. Here is exactly what happens on the IT side.

First, the pipeline triggers. It sees a change to the `main` branch. The pipeline definition (a YAML file in the repo) specifies that anytime someone pushes to `main`, the pipeline should run.

Second, Azure DevOps allocates a Microsoft-hosted agent running Ubuntu. The agent fetches the exact version of the code from the repository, including Sarah's new photo upload feature.

Third, the pipeline executes the 'Restore' task. This runs `npm install` to download all the JavaScript libraries the website needs (like React and Axios). If a library is missing or a version is wrong, the pipeline fails here, and Sarah gets an immediate error log saying 'Package not found'.

Fourth, the pipeline runs 'Build'. It compiles the TypeScript code into plain JavaScript and bundles everything into a few files. This step also lints the code—Linting is a static analysis tool that checks for simple programming errors, style issues, and potential bugs without running the code.

Fifth, the pipeline runs 'Unit Tests'. It executes a suite of 500 automated tests. These tests check that the new 'upload photo' function works correctly: does it accept only image files? Does it reject files larger than 5MB? Does it resize the image correctly? Three tests fail because Sarah forgot to handle the case where a user uploads a non-image file.

Sixth, because the tests fail, the pipeline stops immediately. It does not proceed to 'Publish Artifact'. The output is a red 'Failed' status. An email goes to Sarah with a link to the test output, showing exactly which three tests failed and why. Sarah can fix her code, push the fix, and the pipeline will run again.

Seventh, once the tests pass, the pipeline runs 'Publish Artifact'. It takes the compiled JavaScript files, compresses them into a .zip archive called `shopfast-webapp-20240615.zip`, and stores it in Azure Pipelines. This artifact is now ready for deployment.

In this real scenario, the IT professional (Sarah) doesn't manually do any of this. Her job is to write the pipeline YAML file once, and then the system runs it automatically every time code changes. The benefit is huge: the pipeline caught a bug (the non-image upload issue) before it ever reached production, saving the company from a potential customer complaint and a midnight bug fix.

## Exam focus

The AZ-400 exam tests 'Implement a build pipeline' in several specific ways. You must be ready for scenario-based questions where you are given a description of a development team's needs and you must choose the correct pipeline configuration.

Here is exactly what the exam expects you to know:

- Trigger Types: You must know the difference between CI triggers, PR triggers, scheduled triggers, and manual triggers. A common question gives you a team that wants to run builds on every commit to `feature/*` branches but only on PR merges to `main`. You must configure `trigger` and `pr` sections in YAML correctly.
- YAML Syntax: The exam heavily favours YAML-based pipelines over the classic editor. You must know how to define a `pool` (agent pool), `steps`, `variables`, and `tasks`. A trap question might have a YAML snippet with a syntax error (like a missing colon or incorrect indentation) and ask which change fixes it.
- Build Agents: You must know when to use Microsoft-hosted vs. self-hosted agents. The exam tests the trade-offs: Microsoft-hosted is simpler but has a limited software set and a 60-minute timeout for free-tier accounts. Self-hosted gives you full control but requires maintenance. Questions often present a scenario where the build requires a specific, obscure SDK that is not on the Microsoft-hosted agent, and the correct answer is to set up a self-hosted agent.
- Multi-stage Pipelines: The exam calls them 'multi-stage pipelines', not just 'build pipelines'. You must know how to define multiple stages in one YAML file (e.g., Build, Test, Deploy) and how to use `dependsOn` to control the order. A typical question: 'Your team wants a build to run tests, and if tests pass, deploy automatically. Which YAML construct do you use?' Answer: `stages` with `dependsOn`.
- Pipeline Variables and Variable Groups: Questions will test how to securely inject secrets (passwords, API keys) into a build. You must know that you store secrets in Variable Groups linked to Azure Key Vault, then reference them in the pipeline. Never hard-code a secret in the YAML file. The exam will present a scenario where a build fails because a connection string is missing, and the correct fix is to add a variable group.
- Cache and Restore: You must understand how to speed up builds by caching dependencies (like `node_modules` or `packages`). The exam tests the `Cache@2` task. For example, a question: 'Your .NET build takes 20 minutes because it restores NuGet packages every time. How do you speed it up?' Answer: Add a cache step for NuGet packages.

Commonly tested traps:
- 'Your build succeeds but the artifact is missing.' This means the `PublishPipelineArtifact` or `PublishBuildArtifact` task was either omitted or placed incorrectly (e.g., inside a condition that never evaluated to true).
- 'Your build triggers twice when you push a commit.' This is usually because both the CI trigger and the PR trigger are set, and you forgot to set `trigger: none` on the PR trigger for the same branch.
- 'Your build fails because a tool is not found.' The agent pool is wrong. You need to use an agent that has that tool installed (often a self-hosted agent or a specific Microsoft-hosted agent like `ubuntu-latest` for Linux tools).

Memorise these patterns:
- The default CI trigger: `trigger: none` means no automatic trigger. `trigger: [main]` means trigger on pushes to main.
- The default PR trigger: `pr: none` means no automatic PR trigger. `pr: [main]` means trigger on PRs targeting main.
- Use `pool: vmImage: 'ubuntu-latest'` for Microsoft-hosted Linux agents.
- Use `variables: - group: MyVariableGroup` to import a variable group.

## Step by step

1. **Trigger the Pipeline** — The pipeline starts automatically when a developer pushes code to a configured branch (e.g., `main`) or creates a pull request. This is defined by the `trigger` and `pr` sections in the YAML file. Azure DevOps listens for these events via webhooks.
2. **Allocate a Build Agent** — Azure DevOps selects an available agent from the specified agent pool (e.g., `ubuntu-latest`). The agent is a virtual machine that has the necessary tools (like .NET SDK or Node.js) pre-installed to execute pipeline tasks.
3. **Run Pre-Build Tasks (Restore and Lint)** — The agent runs tasks to prepare the environment. This typically includes restoring dependencies (like `npm install` or `dotnet restore`) and linting the code to catch simple syntax or style issues before compilation.
4. **Compile and Run Tests** — The pipeline compiles the source code into executable output (e.g., using `dotnet build` or `npm run build`). Then it executes automated unit tests (e.g., `dotnet test` or `npm test`). If any test fails, the pipeline stops here and reports 'Failed'.
5. **Publish and Store the Artifact** — If all previous steps succeed, the pipeline packages the compiled output into a deployable artifact (e.g., a .zip file or a Docker image). The `PublishPipelineArtifact` task stores this artifact in Azure Pipelines, making it available for the release pipeline or manual download.

## Comparisons

### CI Trigger vs PR Trigger

**CI Trigger:**
- Triggers on commits to a branch (e.g., push to main)
- Defined using the `trigger` keyword in YAML
- Runs build on the code in the branch after commit

**PR Trigger:**
- Triggers when a pull request is created or updated
- Defined using the `pr` keyword in YAML
- Runs build on the merged code to validate before merging

### Microsoft-hosted Agent vs Self-hosted Agent

**Microsoft-hosted Agent:**
- Managed and maintained by Microsoft
- Pre-installed with common tools (VS, SDKs)
- Limited software flexibility; cannot install arbitrary tools

**Self-hosted Agent:**
- Managed and maintained by you (your own server)
- You install exactly the tools and SDKs you need
- Full control over the environment and security

### YAML Pipeline vs Classic Editor (Visual Designer)

**YAML Pipeline:**
- Defined as code in a repository (azure-pipelines.yml)
- Version controlled alongside source code
- Modern approach recommended by Microsoft

**Classic Editor (Visual Designer):**
- Created and maintained through a web UI in Azure DevOps
- Not stored in source control (only metadata in Azure DevOps)
- Legacy approach, not the focus of the AZ-400 exam

### Pipeline Variable vs Variable Group

**Pipeline Variable:**
- Defined inline in the YAML file or pipeline UI
- Scoped to a single pipeline run
- Can store plain text values or secrets (when marked as such)

**Variable Group:**
- Stored centrally in Azure DevOps
- Can be shared across multiple pipelines
- Often linked to Azure Key Vault for secret management

## Diagram

_Flowchart showing the steps of a build pipeline from code push through restore, lint, compile, test, and publish to artifact._

```mermaid
flowchart TD
    A[Developer pushes code to repo] --> B[Webhook triggers Azure DevOps]
    B --> C[Allocate build agent]
    C --> D[Restore dependencies]
    D --> E[Lint code]
    E --> F[Compile code]
    F --> G[Run unit tests]
    G -- Tests pass --> H[Publish artifact]
    G -- Tests fail --> I[Pipeline fails - email alert sent]
    H --> J[Artifact stored for deployment]
```

## Common misconceptions

- **Misconception:** A build pipeline is only for compiling code into executable files. **Reality:** A build pipeline runs a series of automated tasks, including running tests, checking code quality, scanning for security vulnerabilities, and packaging the output. Compilation is just one possible task. (This misconception comes from the historical term 'build' meaning 'compile'. Modern DevOps pipelines do much more than traditional compiling.)
- **Misconception:** If my code compiles, the build pipeline will always succeed. **Reality:** A build pipeline can fail for many reasons beyond compilation: unit tests fail, linting errors, security scan finds a vulnerability, disk space on the agent runs out, network timeout when downloading dependencies, or a configuration file is missing. (Beginners often equate coding success with test success. They don't realise that a pipeline is a gate with multiple criteria, not just one.)
- **Misconception:** A build pipeline and a release pipeline are the same thing. **Reality:** A build pipeline (continuous integration) takes code and produces an artifact. A release pipeline (continuous delivery/deployment) takes that artifact and deploys it to servers. They are separate stages that can be chained together. (The terms are often used loosely in conversation. The AZ-400 exam defines them as distinct concepts with different purposes and configuration options.)
- **Misconception:** You must use the Azure DevOps classic editor to create a build pipeline. **Reality:** Microsoft recommends and the exam focuses on YAML-based pipelines defined in a file (azure-pipelines.yml) stored in your repository. The classic editor is legacy and not tested as heavily. (Many online tutorials and older documentation still show the classic editor. But real-world DevOps and the exam have moved to the YAML approach.)
- **Misconception:** A build pipeline only runs when someone pushes code to the main branch. **Reality:** You can configure triggers to run on any branch, on pull requests, on a schedule (e.g., nightly builds), on completions of other pipelines, or manually. CI triggers are flexible per branch definition. (Beginners often only see the default 'trigger on main' example and generalise that to all pipelines.)

## Key takeaways

- A build pipeline automates the process of turning source code into a deployable artifact, running tests and quality checks along the way.
- Continuous Integration (CI) means every code commit is automatically built and tested, catching errors early.
- The build pipeline is defined in a YAML file (azure-pipelines.yml) that lives alongside your code in the repository.
- The result of a successful build pipeline is an artifact, which is the packaged output ready for deployment.
- Build agents are the machines that execute pipeline tasks; choose Microsoft-hosted for simplicity or self-hosted for full control.
- Pipeline variables and variable groups allow you to securely inject secrets, connection strings, and configuration values without hard-coding them.
- Multi-stage pipelines allow you to define build, test, and deploy stages in a single YAML file, chained together using the `dependsOn` keyword.
- A failed pipeline stops immediately at the first task that fails, preventing bad code from progressing to deployment.
- Use triggers like CI triggers (push) and PR triggers (pull request) to automatically start a pipeline on specific code events.
- Caching dependencies (e.g., NuGet packages, npm modules) can drastically reduce build times in recurring pipeline runs.

## FAQ

**What is the difference between CI and CD in a build pipeline?**

CI (Continuous Integration) is the automated build and test process that runs every time code is committed. CD (Continuous Delivery or Deployment) is the later step that takes the artifact from CI and deploys it to an environment. The build pipeline handles CI; a separate release pipeline often handles CD.

**What does a YAML pipeline file look like?**

A simple YAML pipeline file starts with `trigger: [main]`, then defines a pool (e.g., `pool: vmImage: 'ubuntu-latest'`), and then a list of steps like `- script: echo Hello World`. It is stored in your repository as `azure-pipelines.yml`.

**Can I run my build pipeline on my own computer?**

Yes, you can set up a self-hosted agent on your own computer or server. This gives you full control over the software installed. However, for the AZ-400 exam, you should know that Microsoft-hosted agents are simpler and managed by Microsoft.

**Why did my build fail even though my code compiles fine locally?**

The build agent environment may differ from your local machine. For example, your local machine might have a different version of a library, or a required tool (like a specific SDK) may not be installed on the agent. Check the pipeline logs to see which step failed and why.

**What is a build artifact and why do I need one?**

A build artifact is the packaged output of a successful build, such as a .zip file or a container image. You need it because it is the single, consistent package you deploy to servers, rather than re-building from source each time.

**What is a variable group and how do I use it in a pipeline?**

A variable group is a collection of variables stored in Azure DevOps that you can link to multiple pipelines. It is used to securely store secrets like API keys or database connection strings. You reference it in your YAML as `variables: - group: MyVariableGroup`.

## Check your understanding

1. **What is the purpose of a CI trigger in a build pipeline?**

   Answer: A CI trigger automatically starts the pipeline when new code is pushed to a specified branch, ensuring that every integration is built and tested.

2. **What is the output of a successful build pipeline called?**

   Answer: The output is called an artifact (e.g., a .zip file or a Docker image) which is stored for later deployment.

3. **If your build requires a specific SDK not available on Microsoft-hosted agents, what should you use?**

   Answer: You should use a self-hosted agent, which you install and configure with the required SDK and tools.

4. **What does the `PublishPipelineArtifact` task do?**

   Answer: It stores the compiled output (artifact) from the build pipeline so it can be used later by a release pipeline or downloaded manually.

5. **A developer pushes code to a `feature/*` branch and wants the build pipeline to run only on that branch, not on `main`. How should the trigger be configured?**

   Answer: Set `trigger: [feature/*]` in the YAML file, and ensure `trigger: none` is not applied.

---

Interactive version with quiz and diagrams: https://courseiva.com/learn/az-400/implementing-a-build-pipeline