# Custom Resource Definitions and Operators

> Chapter 6 of the Courseiva CNCF-CKAD curriculum — https://courseiva.com/learn/cncf-ckad/custom-resource-definitions-and-operators

**Official objective:** 3.2 — Create and manage Custom Resource Definitions (CRDs) and basic Operators

## Introduction

Custom Resource Definitions and Operators let you extend Kubernetes to manage things it wasn't originally designed to handle, like a specific database or a custom application. For the CKAD exam, mastering these tools is critical because they represent a shift from manually configuring applications to having Kubernetes automate their entire lifecycle. Understanding this concept proves you can design scalable, automated systems, which is the heart of modern cloud-native development.

## The Apartment Building's Property Manager Analogy

Because a landlord wants consistent management of unique features in their building, they hire a property manager to define and enforce rules for things beyond standard plumbing or electricity. The landlord (Kubernetes) provides the basic structure: elevators, fire alarms, and central heating. But what about a tenant's request to install a specific type of air conditioner that needs special approval? The landlord doesn't have a built-in system for that specific request. So, the landlord first creates a new "rulebook" — a Custom Resource Definition — for a new thing called "AC Install Request." This rulebook states exactly what information the tenant must provide: the model number, the required voltage, and the installer's license number. The landlord then hires a property manager (the Operator) whose only job is to watch for these new AC Install Request forms. When a tenant submits one, the property manager checks the form against the rulebook. If everything is correct, the property manager doesn't just approve it; they physically go to the basement, flip the correct breaker switch, and ensure the electrical panel has capacity for the new AC unit. This single person handles the entire lifecycle of that request, from receipt to completion and monitoring, ensuring the building's electrical system stays safe. Without this custom rulebook and dedicated manager, the landlord would have to manually approve each strange request, a process that doesn't scale for hundreds of tenants.

## Core explanation

Kubernetes is a powerful system for managing containerised applications. Think of it as a factory floor manager. It comes with a set of standard tools to handle standard machines: you can tell it to run a machine (a Pod), expose it to the network (a Service), or keep a certain number of machines running (a ReplicaSet). But what if you need to manage a very specific machine, like a custom-built database that requires a special startup sequence and regular backups? The standard tools don't understand this machine's unique needs. This is where Custom Resource Definitions (CRDs) and Operators come in.

A Custom Resource Definition (CRD) is the act of creating a new, custom type of object within the Kubernetes API. The API is the set of rules and commands you use to talk to your factory floor manager. Normally, you can only tell it about Pods, Services, and Deployments. A CRD lets you add a new object type, let's call it "MyCustomDatabase." You define its rules in a YAML file. For example, you specify that every MyCustomDatabase object must have fields like `version`, `storageSize`, and `backupSchedule`. Once you apply this CRD to your cluster, you can now create instances of MyCustomDatabase just like you create Pods. In fact, you can use the familiar command `kubectl get mycustomdatabases` to list them. This is a fundamental extension of Kubernetes.

Now you have the ability to declare you want a MyCustomDatabase, but Kubernetes still doesn't know how to bring one to life or keep it healthy. This is where the Operator comes in. An Operator is a custom controller (a self-running program) that watches for the custom objects you defined with the CRD. It is essentially a piece of software that contains all the operational knowledge a human expert would have about your specific database.

Let's break down the Operator's workflow. First, you install the Operator (often as a Deployment) into your cluster. The Operator is programmed to watch the Kubernetes API for events related to your custom resource (MyCustomDatabase). When you use `kubectl apply -f mydb.yaml` to create a new MyCustomDatabase object, the Operator sees this event. It then takes action. Its actions can include: creating the necessary Pod with the correct database image, creating a PersistentVolumeClaim for storage, setting up a network Service for access, and starting the initial database backup schedule. The Operator doesn't stop after creation. It continuously monitors the state of the database. If the database Pod crashes, the Operator can automatically recreate it with the correct configuration. If you update the MyCustomDatabase object to request a larger storage size, the Operator might automatically trigger a storage expansion. This is the key difference: an Operator manages the entire lifecycle of an application, not just deployment.

Without this pattern, you would have to write complex scripts and CronJobs to handle these tasks manually. You would have to write a script to create the Pod, another to set up the backups, and another to handle failures. You are essentially building an application that runs inside Kubernetes to manage another application. Why does this exist? It exists to automate the work of a skilled human operator. In large-scale systems, you cannot have a person manually configuring and restarting databases every time something fails. The Operator codifies that expertise into a program that never sleeps.

The CKAD exam focuses on the practicalities of working with CRDs and a basic Operator, not on writing your own Operator from scratch. You need to understand how to create a CRD from a YAML manifest. The exam expects you to know the structure of a CRD, specifically the `spec.names` and `spec.scope` fields. The `scope` field determines if your custom resource is namespaced (exists within a single namespace) or cluster-scoped (global). You also need to understand how to create an instance of the custom resource by using your CRD's definitions. The exam might ask you to deploy a pre-built Operator from a Helm chart or a set of YAML files. The critical skill is understanding the relationship: the CRD defines the blueprints, and the Operator is the builder that brings those blueprints to life and maintains them.

## Real-world context

An IT professional at a mid-sized company is tasked with managing a MongoDB cluster for a critical financial application. The traditional approach would be to manually install MongoDB on a virtual machine, configure replica sets, set up monitoring, and create backup scripts. This is error-prone and does not scale. Instead, the engineer decides to use a Kubernetes Operator for MongoDB.

The engineer begins by installing the MongoDB Operator. This is usually done by applying a set of YAML files or using a Helm chart. Once the Operator's Pod is running, it is actively watching the Kubernetes API for a specific custom resource, typically named something like `MongoDB` or `MongoDBCluster`. The engineer then creates a YAML file that defines their desired MongoDB cluster state. This file is an instance of the custom resource. It might look like this:

- apiVersion: mongodb.com/v1
- kind: MongoDB
- metadata:
-   name: financial-db
- spec:
-   version: "6.0.5"
-   members: 3
-   backupConfiguration:
-     enabled: true
-     schedule: "0 2 * * *"

When the engineer runs `kubectl apply -f financial-mongodb.yaml`, the Operator sees this new object. Because the Operator is programmed with MongoDB expertise, it does the following:

- It automatically creates a Kubernetes StatefulSet (a workload resource for stateful applications) with three Pods, one for each member of the MongoDB Replica Set.
- It creates a Headless Service for database discovery.
- It configures the Pods to talk to each other and form a Replica Set.
- It sets up a CronJob to automatically perform backups at 2 AM every day as specified.
- It creates a Secret for the root username and password.

Six months later, the finance team needs to increase the database storage. The engineer simply edits the MongoDB custom resource YAML file, changing a `storageSize` field, and re-applies it: `kubectl apply -f financial-mongodb.yaml`. The Operator detects the change and automatically triggers a storage expansion operation on the underlying PersistentVolumeClaim, a task that would normally require manual database intervention.

A few weeks later, a node fails and one of the MongoDB Pods is destroyed. The Operator's reconciliation loop detects the mismatch between the desired state (3 healthy members) and the actual state (2 healthy members). The Operator automatically creates a new Pod on a healthy node and configures it to re-join the Replica Set, ensuring zero data loss and minimal downtime. The human operator is alerted but does not need to act. This is the value proposition: the Operator handles complex, recurring operational tasks, freeing the IT professional to work on higher-level features instead of fixing production databases at 3 AM.

## Exam focus

The CKAD exam (version 2024 and later) tests your ability to **use** CRDs and a basic Operator, not to design an Operator from scratch. The exam is practical and task-based. You will be given a scenario and must execute commands in a terminal. Here is what you must be prepared for.

**Question Type 1: Creating and applying a CRD.** You will be given a specification and asked to create a YAML file for a new CRD. The exam loves testing the `spec.names` hierarchy. You must memorise the following fields and their purposes:

- `spec.names.plural`: The name used in `kubectl get ...`. Always lowercase.
- `spec.names.singular`: The name used for the resource type in `kubectl`.
- `spec.names.kind`: The value used in the `kind` field of your custom resource instance. Usually starts with a capital letter.
- `spec.scope`: Must be either `Namespaced` or `Cluster`. A common trap is setting scope to `Namespaced` when the question implies a cluster-scoped resource.

**Question Type 2: Creating an instance of a custom resource.** After the CRD exists, you will be asked to create an object using that CRD. The trap here is that you must match the `apiVersion` (which is `spec.group`/`spec.version` from the CRD) and the `kind` (which is `spec.names.kind`). If you get these wrong, the API will reject your resource.

**Question Type 3: Installing and using a basic Operator.** The exam might provide you with the YAML files for an Operator (often a Deployment) and its CRD. You need to apply them in the correct order: first the CRD, then the Operator's own resources (ServiceAccount, Role, RoleBinding, Deployment), then the custom resource instance. A trap is that you might try to create the custom resource instance before the Operator is running, which is fine (the Operator will reconcile later), but the CRD must exist first.

**Question Type 4: Verifying the Operator's work.** After creating the custom resource, the question may ask you to verify that the Operator has created the expected resources (like Pods or Services). You must know how to search for Pods with labels that the Operator typically adds. For example, you might run `kubectl get pods -l app=<custom-resource-name>`.

**Traps to avoid:**
- Forgetting to include the `apiVersion` field in your custom resource instance. This must match the `spec.group` and `spec.version` in the CRD.
- Using incorrect casing for the `kind` field. If the CRD defines `kind: MyApp`, you must write `kind: MyApp` exactly.
- Confusing `spec.names.kind` with `spec.names.plural`. The `kind` is used inside the resource YAML; the `plural` is used with `kubectl`.
- Assuming an Operator automatically creates a Service for your custom resource. Not all Operators do this; you must check the documentation or the question's instructions.

**Key definitions to memorise:**
- CRD (CustomResourceDefinition): A way to extend the Kubernetes API by defining a new object type.
- Custom Resource (CR): An instance of a CRD, like creating a specific Pod from a Pod template.
- Operator: A custom controller that uses one or more CRDs to automate the lifecycle of an application.
- Reconciliation Loop: The endless loop an Operator runs to compare the actual state of the system to the desired state (defined in the CR) and make changes to align them.

## Step by step

1. **Define the Custom Resource Definition YAML** — Create a YAML file that specifies the new object type. This file must include the apiVersion (apiextensions.k8s.io/v1), kind (CustomResourceDefinition), and the spec section with metadata.name, spec.group, spec.names (plural, singular, kind), spec.scope, and spec.versions. This is the blueprint for your new resource.
2. **Apply the CRD to the Cluster** — Run `kubectl apply -f <crd-file.yaml>`. This registers the new object type with the Kubernetes API. After this step, running `kubectl get <plural-name>` will work but return no resources until instances are created. The CRD alone does not run anything.
3. **Install the Operator's Prerequisites** — Install the Operator's ServiceAccount, Role, and RoleBinding (or ClusterRole and ClusterRoleBinding) using separate YAML files. The Operator needs these permissions to watch for CRD instances and manage resources like Pods and Services. Apply these before the Operator Deployment.
4. **Deploy the Operator as a Deployment** — Create a Deployment YAML that runs the Operator's container image. Apply this YAML to the cluster. The Operator's controller will start and begin watching the API for custom resource instances. It is now ready to react to your desired state.
5. **Create an Instance of the Custom Resource** — Write a YAML file that uses the kind and apiVersion defined in your CRD. Fill in the spec fields according to your CRD's schema. Apply this file with `kubectl apply -f <instance.yaml>`. The Operator's reconciliation loop will detect this new object and take action, such as creating Pods or configuring backups, to bring the system to the desired state.

## Comparisons

### CRD (Custom Resource Definition) vs Custom Resource Instance

**CRD (Custom Resource Definition):**
- Defines the schema for a new object type.
- Applied once to register the type with the API.
- Analogous to a class definition in programming.

**Custom Resource Instance:**
- An actual object that conforms to a CRD's schema.
- Applied multiple times to declare desired state.
- Analogous to an object instance in programming.

### Operator vs Helm Chart

**Operator:**
- A running program (controller) that manages application lifecycle.
- Watchs for changes and reacts automatically.
- Requires a controller (Pod) running 24/7.

**Helm Chart:**
- A static package of YAML templates for deploying applications.
- Used only at installation or upgrade time.
- Does not run any program after deployment; it is a set of files.

### Namespaced CRD vs Cluster-Scoped CRD

**Namespaced CRD:**
- Resources exist within a single namespace.
- Isolated from other namespaces.
- Uses spec.scope: Namespaced.

**Cluster-Scoped CRD:**
- Resources exist globally across all namespaces.
- Can be referenced from any namespace.
- Uses spec.scope: Cluster.

## Diagram

_This diagram shows the flow from defining a CRD, to creating a custom resource instance, to the Operator's reconciliation loop that creates the necessary infrastructure._

```mermaid
flowchart TD
    A[User applies CRD YAML] --> B[CRD registered with Kubernetes API]
    B --> C[User applies Custom Resource YAML]
    C --> D[Operator watches API for new resource]
    D --> E{Operator's Reconciliation Loop}
    E --> F[Operator creates Pods, Services, ConfigMaps]
    F --> G[Actual State matched to Desired State]
    G --> E
```

## Common misconceptions

- **Misconception:** An Operator is the same thing as a Helm chart. **Reality:** A Helm chart is a package of Kubernetes YAML files for deploying applications. An Operator is a running program that manages the lifecycle of an application. You can use a Helm chart to install an Operator. (Both are used to deploy applications, but people confuse a static template (Helm) with an active, intelligent controller (Operator) that reacts to changes.)
- **Misconception:** You must write your own custom Operator code to pass the CKAD exam. **Reality:** The CKAD exam expects you to use pre-built Operators and understand their relationship to CRDs. You do not need to write Go or Python code for an Operator. (The name 'Operator' sounds like a complex programming task, so beginners fear they need to be software engineers. The exam tests usage, not coding.)
- **Misconception:** A CRD automatically manages the resources it defines. Once you create a CRD, Kubernetes knows how to run a database. **Reality:** A CRD only defines the structure for a new object type. It does not include any logic to create Pods or manage state. The Operator provides the logic. (People think 'definition' implies control. They don't realise a CRD is just a data schema (a blueprint) and needs a separate controller (the Operator) to act on it.)
- **Misconception:** All Operators create Pods. If you create a custom resource, you will always see a Pod appear. **Reality:** An Operator can manage any resource, including external services. For example, an Operator might create a DNS record in a cloud provider or provision a managed database outside the cluster. (Because the most common example Operators (like for MySQL or MongoDB) create Pods, beginners assume this is the only behaviour. They don't grasp the general-purpose nature of Operators.)

## Key takeaways

- A Custom Resource Definition (CRD) extends the Kubernetes API by defining a new object type; it is just a schema and contains no logic.
- An Operator is a custom controller that watches for changes to CRD instances and automates the lifecycle of an application, turning manual operations into code.
- The CKAD exam will ask you to create a CRD YAML file, not write an Operator from scratch, and you must memorise the spec.names fields (plural, singular, kind).
- When creating a custom resource instance, the apiVersion must match the group and version from the CRD, and the kind must match exactly as defined in the CRD's spec.names.kind.
- Operators use a reconciliation loop to continuously compare the desired state in a custom resource with the actual state in the cluster and take action to align them.
- To pass the exam, practise the order of operations: apply the CRD first, then the Operator's service account and roles, then the Operator deployment, and finally the custom resource instance.
- A common exam trap is setting the CRD scope to 'Namespaced' when the resource should be cluster-scoped, or vice versa; read the question carefully.
- You can use the command kubectl get customresourcedefinitions to list all CRDs in the cluster, and kubectl get <plural-name> to list instances of a specific CRD.

## FAQ

**What is the difference between a Kubernetes CRD and a custom resource?**

A CRD is the definition or blueprint for a new object type you create using a YAML file. A custom resource is an actual instance of that type, similar to how a Pod is an instance of a Pod specification.

**Do I need to write my own Operator to pass the CKAD exam?**

No. The CKAD exam tests your ability to deploy and use pre-built Operators and apply CRDs. You will not be asked to write Go or Python code for an Operator.

**What does the `spec.scope` field in a CRD mean?**

It determines whether your custom resource is namespaced (exists within a single namespace) or cluster-scoped (global across all namespaces). The value must be either `Namespaced` or `Cluster`.

**If I create a CRD, does Kubernetes automatically manage the application it defines?**

No. A CRD only defines the data structure. You need an Operator (a custom controller) to watch for instances and perform actions like creating Pods or managing backups.

**What is a reconciliation loop in the context of an Operator?**

It is a continuous loop the Operator runs. It reads the desired state from your custom resource instance, checks the actual state of the cluster, and takes actions (like creating or deleting Pods) to make the actual state match the desired state.

**Can I use `kubectl get crd` to list custom resource definitions?**

Yes, but the full command is `kubectl get customresourcedefinitions` or the shorthand `kubectl get crd`. It lists all CRDs registered in the cluster.

## Check your understanding

1. **What is the purpose of the `spec.names.kind` field in a Custom Resource Definition YAML?**

   Answer: It defines the value you use in the `kind:` field of your custom resource instance YAML file, such as `MyApp` or `Database`.

2. **You create a CRD with `spec.scope: Namespaced`. In which namespace can you create an instance of this custom resource?**

   Answer: You can create it in any namespace, and the resource will belong to that specific namespace.

3. **After applying a CRD, you run `kubectl get pods` but see no Pods. You have not applied any custom resource instance yet. Is this correct?**

   Answer: Yes, a CRD only defines the type; the Operator creates Pods only when a custom resource instance is applied and the Operator is running.

4. **What command would you use to see all objects of a custom resource type whose plural name is `myapps`?**

   Answer: `kubectl get myapps` or `kubectl get myapps -n <namespace>` if namespaced.

5. **On the CKAD exam, you are asked to ensure a custom resource instance is automatically managed. What must you deploy in addition to the CRD?**

   Answer: You must deploy an Operator, which is a controller that watches for the custom resource and performs lifecycle management actions.

---

Interactive version with quiz and diagrams: https://courseiva.com/learn/cncf-ckad/custom-resource-definitions-and-operators