# Building CI Pipelines with Cloud Build

> Chapter 5 of the Courseiva GOOGLE-PCDOE curriculum — https://courseiva.com//learn/google-pcdoe/building-ci-pipelines-with-cloud-build

**Official objective:** 1.3 — Design and implement continuous integration workflows using Cloud Build triggers and build configurations.

## Introduction

If you skip understanding how to automate your builds, you will spend half your day manually running tests, copying files, and praying you didn't miss a step. That is exactly the problem continuous integration (CI) solves — it takes the boring, repetitive work of building and testing software and makes it happen automatically every time you or your team pushes new code. For the Google Professional Cloud DevOps Engineer exam, knowing how to design these automated pipelines using Cloud Build is one of the most heavily tested skills because it is the foundation of delivering reliable software at speed.

## The Assembly-Line Recipe Book Analogy

An industrial kitchen at dawn, with glass walls and stainless-steel counters.

Every morning the head chef walks to a stand that holds a huge, spiral-bound book — the recipe book. This book doesn't just list ingredients; it gives exact steps, temperatures, and timings for every dish. When the chef turns to the page for 'Sourdough Loaf', they don't need to remember anything. They follow the book: mix flour and water, add starter, wait forty minutes for the autolyse step, then fold the dough. The book tells the assistant which bowl to use, tells the oven operator the exact temperature (220°C), and even says when to open the steam vent.

Now imagine the chef has a special camera that watches the ingredients bin. When a new bag of flour is delivered and stacked on the shelf (a trigger event), the camera sends a notification to the chef. The chef immediately opens the recipe book to the 'Bread' section and starts the entire process automatically. No one needs to shout "New flour!" — the system sees the delivery and runs the instructions.

In this analogy, the recipe book is the Cloud Build configuration file (cloudbuild.yaml), the camera and notification system is the Cloud Build trigger (watching a source repository), and the kitchen workers are the build steps. The chef never deviates from the book, ensuring every loaf is identical and perfectly baked, just like CI pipelines ensure every code change produces the same tested artefact.

## Core explanation

Let us start from scratch. When a developer writes code, that code is just text in a file. Before it can become a working application that users can interact with, that text needs to be 'built'. Building means taking the raw source code and turning it into a runnable package — for example, compiling Java code into a .jar file, or bundling JavaScript files into a single minified script, or creating a container image that holds your application and everything it needs to run.

Traditionally, developers would run these build steps on their own laptops. They would open a terminal, type commands, wait for the build to finish, and then manually upload the result to a server. This approach has three giant problems. First, the build might work on my laptop but fail on yours because our tools are slightly different versions — this is the famous 'it works on my machine' problem. Second, it is slow and wastes developer time that could be spent writing code. Third, nothing stops a developer from accidentally skipping a step, like forgetting to run the unit tests before deploying.

Continuous integration (CI) solves all three. CI is a practice where every time someone sends new code to a shared repository — like GitHub, GitLab, or Google Cloud Source Repositories — a server automatically picks up that code, builds it, runs tests against it, and reports back whether it passed or failed. No human babysitting required. Cloud Build is the Google Cloud service that provides this automation.

A Cloud Build pipeline is defined in a file called cloudbuild.yaml. Think of this file as a recipe. It sits in the root of your source code repository. Inside the file, you list a series of 'steps'. Each step runs a specific command inside a container — a lightweight, isolated environment that has exactly the tools needed. For example, your first step might run 'npm install' to download dependencies for a Node.js application. The second step might run 'npm test' to execute your test suite. The third step might run 'docker build' to create a container image of your application. The fourth step might push that image to Google Container Registry or Artifact Registry.

But how does Cloud Build know when to start? This is where triggers come in. A Cloud Build trigger is a rule that says: 'When X happens in my repository, run this build configuration'. The trigger is linked to a source repository and a branch. For example, you can set up a trigger that fires whenever someone pushes code to the 'main' branch. You can also set up triggers for pull requests — so that every time a developer opens a pull request, Cloud Build builds the code and runs tests before anyone merges anything. This catches bugs early, before they ever reach production.

Cloud Build supports several source repositories: Google Cloud Source Repositories (Google's own hosted git service), GitHub, Bitbucket, and GitLab. The trigger configuration includes details like which repository, which branch (or branches), and even which files changed. You can use regular expressions to match file paths, so you only trigger a build if, say, the 'src/' directory changes, ignoring documentation edits.

Why does all of this matter for the PCDOE exam? Because the exam will test you on exactly two things: (1) the syntax and structure of a cloudbuild.yaml file, including how to define build steps, substitutions (variables you can pass into the build), and which images or artefacts to produce; and (2) how to configure triggers, including using regular expressions for branch patterns, setting up pull-request builds, and connecting external repositories like GitHub.

You will also need to understand the concept of 'build config as code'. This means your pipeline definition lives right alongside your application code in the same repository. If someone changes the pipeline, it gets versioned and reviewed just like normal code changes. This is a key DevOps principle — treat your infrastructure and configuration the same way you treat application code.

Cloud Build is serverless, which means you do not need to manage any virtual machines or clusters to run your builds. Google spins up the underlying compute resources on demand, runs your steps inside containers, and tears everything down when done. You only pay for the time the build runs. This is a huge advantage over older systems where you had to maintain a dedicated build server.

One final critical concept: artefacts. When a build finishes, it produces something — maybe a Docker image, a compiled binary, a .zip file, or a set of test results. Cloud Build can store these artefacts in Google Cloud Storage or push them to Artifact Registry. You can also configure Cloud Build to automatically deploy the built artefact to a Google Cloud service like Cloud Run, Google Kubernetes Engine (GKE), or App Engine. This is called continuous delivery — the 'CD' part of CI/CD.

## Real-world context

Let me walk you through a realistic scenario at a company called 'QuickCart', an e-commerce startup that sells handmade candles online. QuickCart has five developers, all working on the same Node.js backend application. They use GitHub to host their code. Every day, each developer works on a separate 'feature branch' — for example, 'add-express-checkout' or 'fix-cart-bug'.

Before adopting Cloud Build, the QuickCart team had a nightmare. Every Friday afternoon, they did a manual release. Sarah would pull everyone's latest changes onto her laptop, try to build the project, and almost always fail because someone had installed a new npm package that she did not have locally. Then she would spend two hours debugging. Sometimes she forgot to run the linter or the integration tests, and a bug would slip into production on Monday morning, causing the checkout page to crash. Customers were furious.

The QuickCart team decided to implement a CI pipeline with Cloud Build. Here is exactly what they did, step by step:

- First, they added a cloudbuild.yaml file to the root of their repository. The file contained these steps: install dependencies using 'npm ci', run the linter using 'npm run lint', run unit tests using 'npm test', build the Docker image using 'docker build', push the image to Artifact Registry, and finally deploy the new image to Cloud Run using the 'gcloud run deploy' command.

- Second, they connected their GitHub repository to Cloud Build using the Google Cloud console. They authorised Google Cloud to access their GitHub organisation. This connection took about ten minutes.

- Third, they created two triggers. The first trigger was named 'CI-PR'. It fired on every pull request against the 'main' branch. This trigger only ran the first three steps (install, lint, test) — it did not build the Docker image or deploy anything. The idea was to give fast feedback to developers. The second trigger was named 'CI-CD-Main'. It fired on every push to the 'main' branch. This trigger ran all six steps, culminating in a deployment to a staging environment (a second Cloud Run service).

- Fourth, they added a 'branch filter' using a regular expression: the PR trigger only matched branch names like 'feature-.*', so that changes to documentation files (in the 'docs/' folder) did not trigger an unnecessary build.

- Fifth, they configured Cloud Build to send Slack notifications. When a build failed, the team got a message in their '#build-alerts' channel with a link to the failing step logs. This meant Sarah did not have to monitor the builds manually.

- Sixth, they set up 'build substitutions' — variables like '$_REGION' and '$_SERVICE_NAME' that they could change without editing the cloudbuild.yaml file. This allowed them to use the same configuration file for both staging and production, only changing the substitution values in the trigger settings.

The result? Pull requests now get built and tested in under three minutes. Developers get a green checkmark or red X directly on their GitHub pull request page, so they know immediately if their change broke something. The Friday release ritual is gone. Deployments happen automatically whenever code is merged to main, and the team has not had a single deployment-related outage in six months.

What does Sarah, the senior developer, actually do now? Her mornings are spent reading code reviews and planning features, not fighting build tools. If a build fails, she clicks the link in Slack, looks at the logs in Google Cloud Console, and fixes the issue. The logs show her which step failed and the exact error message. She can also re-run a failed build with the same commit by clicking 'Retry' in the Cloud Build console — no need to push a new commit.

The team also learned a painful lesson: they initially used 'npm install' instead of 'npm ci' in their build steps. The difference is subtle but important. 'npm install' can produce different results on different machines because it updates the package-lock.json file. 'npm ci' is deterministic — it uses exactly the versions in the lock file. This fixed the 'works on my machine' problem entirely.

## Exam focus

The PCDOE exam tests Cloud Build heavily. Based on the official exam guide and community feedback, here is exactly what you need to know and how the exam tries to trick you.

First, the exam expects you to read a cloudbuild.yaml file and identify errors or missing required fields. The most common traps involve the 'artifacts' and 'images' sections. For example, if a pipeline builds a Docker image, the cloudbuild.yaml must include an 'images' field listing the image name, or alternatively use the 'docker push' command as a step. The exam might show you a config that builds the image but never pushes it, and they will ask 'what is missing?' Alternatively, they might show a config with an invalid YAML structure — missing colons or incorrect indentation.

Second, triggers are a hot topic. Expect multiple-choice questions where you need to choose the correct trigger configuration for a specific scenario. For instance: 'A team wants to run a build only when code is pushed to branches matching 'release-*' and only when files in the 'backend/' directory change.' The correct answer involves setting both the branch filter (using a regex like '^release-.*$') and an inclusion filter for the file path. The trap is using the wrong filter type — there is a difference between 'includedFiles' and 'excludedFiles' in the trigger YAML. The exam loves testing this distinction.

Third, you must understand how to handle secrets and sensitive data in builds. Cloud Build has a feature called 'Secret Manager' — you can store API keys or passwords in Google Cloud Secret Manager and then reference them in your build steps. The exam will test that you do NOT put secret values directly in cloudbuild.yaml (a common trap). Instead, you use a 'availableSecrets' block to reference the secret from Secret Manager. Similarly, using build substitutions for secrets is also a bad practice because substitutions are visible in the console logs.

Fourth, the exam tests your knowledge of build status reporting. Cloud Build can send notifications to Pub/Sub, Slack, or email. You might get a question like: 'A team wants to automatically trigger a different pipeline when a build fails. Which feature allows this?' The answer is Cloud Pub/Sub notifications, because you can subscribe to the 'cloud-builds' Pub/Sub topic and route failure messages to another Cloud Function or Cloud Run service.

Fifth, you need to understand the build lifecycle and the 'timeout' field. The exam might present a scenario where a build runs longer than the default timeout (10 minutes). You need to know how to increase it both in the cloudbuild.yaml file (using the 'timeout' field) and at the trigger level. There is also a 'substitutions' concept: predefined substitutions like $PROJECT_ID, $BUILD_ID, $REPO_NAME, and $BRANCH_NAME are automatically available inside your build steps. The exam will test which substitutions are built-in versus which ones you need to define yourself.

Here is a list of the specific concepts the exam requires you to memorise: 

- The structure of cloudbuild.yaml: 'steps', 'images', 'artifacts', 'timeout', 'options', 'substitutions', 'tags', 'serviceAccount', and 'logsBucket'. 
- The difference between a push trigger and a pull-request trigger. 
- The syntax for branch filters and file-change filters. 
- How to use 'gcloud builds submit' from the command line (for manual builds). 
- The 'kaniko' cache feature for speeding up container builds. 
- How to use Cloud Build with Cloud Run, GKE, and Compute Engine deploy steps. 
- The purpose of 'logsBucket' for storing build logs beyond the default 30-day retention. 
- How to use 'waitFor' in build steps to control the order of execution. 
- The fact that Cloud Build runs steps in the order they are defined in the YAML file by default, but you can parallelise steps using waitFor: ['step-name']. 

Exam trap patterns: 
- They show a trigger configured to fire on 'any branch' but the dev team only wants 'main' — the answer is to set a branch filter to ^main$. 
- They show a cloudbuild.yaml that omits the 'images' field when building a Docker image — the answer is that the image will be built but not pushed anywhere. 
- They give a scenario where a build fails due to insufficient permissions — the correct fix is to use a custom service account with the necessary IAM roles, not to edit the cloudbuild.yaml steps. 
- They ask about running local builds before committing — the correct tool is 'cloud-build-local', which runs the same steps on your laptop for testing. 

Finally, the exam expects you to understand the relationship between Cloud Build and other Google Cloud services like Cloud Source Repositories, Artifact Registry, Container Registry (legacy), and IAM. You will need to know which permissions are required for Cloud Build to access a GitHub repository (Cloud Build service account needs 'Cloud Build Service Account' role).

## Step by step

1. **1. Connect Your Source Repository** — Go to the Cloud Build page in Google Cloud Console, click on 'Triggers', and select 'Connect Repository'. Choose your provider (GitHub, GitLab, Bitbucket, or Google Cloud Source Repositories). Follow the authorisation prompts to give Cloud Build read access to your repository. This creates a secure connection that Cloud Build uses to listen for events like push or pull request.
2. **2. Create a Build Configuration File (cloudbuild.yaml)** — Create a file called cloudbuild.yaml in the root of your repository. Define the build steps: each step specifies a 'name' (the container image to use, e.g., 'node:16'), an 'entrypoint' (the command to run), and 'args' (the arguments for that command). Example: a step that runs 'npm ci' and 'npm test'. This file is the blueprint for the pipeline.
3. **3. Create a Build Trigger** — In the Cloud Build console, click 'Create Trigger'. Give it a name, choose the connected repository and branch (e.g., '^main$' for pushes to main). Optionally, set inclusion or exclusion file filters so that the build only runs when specific files change. Choose whether to use the cloudbuild.yaml at the repository root or specify a custom path. Save the trigger.
4. **4. Push Code to Test the Trigger** — Make a change to your code, commit it, and push it to the branch you configured. Within seconds, Cloud Build will detect the push, start a new build, and run each step defined in cloudbuild.yaml. You can watch the build progress in real time in the Cloud Build console under 'History'. Each step shows logs: standard output and error messages.
5. **5. Review Build Results and Notifications** — After the build completes, check the status (SUCCESS or FAILURE). If it fails, click on the build ID to see logs for each step and identify the error. Optionally, set up notifications via Slack, email, or Pub/Sub so your team gets alerts automatically. The build result is also reported back to GitHub as a check on your pull request or commit.

## Comparisons

### Push Trigger vs Pull Request Trigger

**Push Trigger:**
- Fires when code is pushed to a branch (e.g., main, feature-branch).
- Typically used to build and deploy to staging or production.
- Can be configured to skip builds for certain branches using branch filters.

**Pull Request Trigger:**
- Fires when a pull request is opened, updated, or reopened against a target branch.
- Typically used to run tests and linting without deploying.
- Results are shown as a check on the pull request page, blocking merge if it fails.

### cloudbuild.yaml (Inline Config) vs Trigger Config (Console Settings)

**cloudbuild.yaml (Inline Config):**
- Stored in the repository alongside code, versioned and reviewable.
- Defines the build steps, images, artefacts, and timeout.
- Must be present in the repository for the trigger to use it.

**Trigger Config (Console Settings):**
- Configured in the Cloud Build console or via API/CLI.
- Defines which repository, branch, and file paths trigger the build.
- Can override or add build substitutions, but cannot change the build steps.

### Predefined Substitutions vs Custom Substitutions

**Predefined Substitutions:**
- Built-in variables like $PROJECT_ID, $BUILD_ID, $REPO_NAME.
- Cannot be overridden or changed in cloudbuild.yaml.
- Always available without any configuration.

**Custom Substitutions:**
- Defined by you in the trigger settings or via --substitutions flag.
- Must start with an underscore (e.g., $_MY_VAR).
- Used for passing environment-specific values like region or service name.

## Common misconceptions

- **Misconception:** Cloud Build only works with Google Cloud Source Repositories and cannot connect to GitHub, GitLab, or Bitbucket. **Reality:** Cloud Build can connect to GitHub (including GitHub Enterprise), GitLab, Bitbucket, and Google Cloud Source Repositories using the console or the 'gcloud' command-line tool. You just need to authorise the connection once. (Beginners often assume Google Cloud services only work with Google's own tools, but Cloud Build has first-class support for external git providers through mirroring or direct integration.)
- **Misconception:** If I define build steps in cloudbuild.yaml, they must all run sequentially in the order listed. **Reality:** By default, steps do run in order, but you can control this using the 'waitFor' field. If step A has 'waitFor: ['-']' (a hyphen), it starts immediately after any previous step, but you can also set steps to run in parallel by specifying that they wait for different previous steps. (Many people are used to traditional CI tools like Jenkins where pipelines are strictly sequential. Cloud Build's 'waitFor' system is more flexible but it is not obvious to newcomers.)
- **Misconception:** Cloud Build automatically stores all build logs forever so I don't need to worry about log retention. **Reality:** By default, build logs are stored for 30 days. If you need longer retention, you must configure a 'logsBucket' in your project settings or inside the cloudbuild.yaml file to store logs in a Cloud Storage bucket of your choice. (The UI displays logs for recent builds, so beginners assume they are permanent. The exam tests this distinction explicitly.)
- **Misconception:** Build substitutions like $PROJECT_ID are customisable and I can change them in my cloudbuild.yaml file. **Reality:** $PROJECT_ID, $BUILD_ID, $REPO_NAME, $BRANCH_NAME and a few others are 'predefined substitutions' that Cloud Build sets automatically. You cannot override them in cloudbuild.yaml; you can only use them as they are. Custom substitutions must use a different name and be defined in the trigger or via the '--substitutions' flag. (Beginners see the dollar-sign syntax and assume all variables work like environment variables they can set themselves. The exam uses this confusion to set traps.)
- **Misconception:** A Cloud Build trigger must point to a specific cloudbuild.yaml file in the repository root. **Reality:** You can specify a custom path to the build configuration file using the 'cloudbuild.yaml' location field in the trigger. It does not have to be at the root — it can be anywhere in the repository, and you can even have multiple config files (e.g., one for testing, one for production). (Most tutorials put cloudbuild.yaml in the root, so beginners think that is a requirement. The exam tests the flexibility of the location field.)

## Key takeaways

- A Cloud Build pipeline is defined in a cloudbuild.yaml file that contains a list of steps, each step runs a command inside a container, and the order is controlled by the 'waitFor' field.
- A Cloud Build trigger watches a source repository (e.g., GitHub) for events like pushes to specific branches or pull requests, and automatically starts a build when the event matches the trigger's filters.
- The 'images' field in cloudbuild.yaml is required if you want the built Docker image to be pushed to Container Registry or Artifact Registry; without it, the image is built but remains only in the local build environment.
- Predefined substitutions like $PROJECT_ID, $BUILD_ID, and $BRANCH_NAME are automatically available inside build steps and cannot be overridden.
- You can parallelise build steps by using the 'waitFor' field with a list of step IDs, and you can make a step start immediately by setting 'waitFor: ['-']'.
- Build logs are retained for 30 days by default; to keep logs longer, you must configure a 'logsBucket' in Cloud Storage.
- Cloud Build supports connecting to GitHub, GitLab, Bitbucket, and Google Cloud Source Repositories through simple authorisation flows in the console.
- You can use 'availableSecrets' in cloudbuild.yaml to reference sensitive data from Secret Manager, keeping secrets out of the pipeline configuration file.
- The 'timeout' field in cloudbuild.yaml sets the maximum duration for a build; an empty timeout defaults to 10 minutes.
- Builds can be triggered manually using the 'gcloud builds submit' command, which accepts a cloudbuild.yaml file or a directory.

## FAQ

**Do I need to pay for Cloud Build if I only run a few builds per day?**

Google Cloud offers a free tier for Cloud Build that includes 120 build-minutes per day for certain machine types. Many small teams never exceed this. Beyond the free tier, you pay per build-minute used, and builds that finish quickly cost very little.

**Can I use Cloud Build to deploy to services other than Cloud Run?**

Yes. Cloud Build can deploy to Google Kubernetes Engine (GKE), App Engine, Compute Engine (via instance templates or SSH), Firebase, and Cloud Functions. You just need to include the appropriate deployment command (e.g., 'gcloud app deploy', 'gcloud functions deploy') in a build step.

**Do I need to install Docker on my own machine to create Cloud Build pipelines?**

Not at all. Cloud Build runs all steps inside containers on Google Cloud's infrastructure. You only need a text editor to write the cloudbuild.yaml file and the 'gcloud' CLI if you want to submit builds from your terminal. Docker is not required on your machine.

**What happens if my cloudbuild.yaml file has a syntax error?**

Cloud Build will reject the build and show an error message like 'Invalid build configuration file' or 'YAML parse error'. The build will fail immediately without running any steps. You can fix the file, commit the fix, and push again.

**Can I run a build locally before pushing to test my cloudbuild.yaml?**

Yes. Google provides a tool called 'cloud-build-local' (part of the 'cloud-build-local' package) that runs the same steps as Cloud Build would, but on your own machine. This is useful for debugging your configuration before committing.

**How do I pass environment variables to my build steps?**

You can define environment variables inside each step using the 'env' field in cloudbuild.yaml. You can also use build substitutions (e.g., '$_MY_VAR') that are set in the trigger configuration or passed via the '--substitutions' flag when using 'gcloud builds submit'.

---

Interactive version with quiz and diagrams: https://courseiva.com//learn/google-pcdoe/building-ci-pipelines-with-cloud-build