# Health Checks: Readiness, Liveness, and Startup Probes

> Chapter 8 of the Courseiva CNCF-CKAD curriculum — https://courseiva.com/learn/cncf-ckad/readiness-and-liveness-probes

**Official objective:** 4.2 — Configure and use readiness, liveness, and startup probes

## Introduction

Exam objective 4.2 is all about keeping your applications running reliably, which is the heart of what a CKAD-certified developer does. This concept of health checks solves a critical problem: how to know if your app is actually working — not just started, but ready to do its job — and what to do if it breaks. For your CKAD exam, you need to master three types of probes — startup, liveness, and readiness — because they are the difference between an app that crashes silently and one that heals itself automatically.

## The Pop-Up Restaurant Dinner Service Analogy

Have you ever run a temporary pop-up restaurant for one night only? You've got a tiny kitchen and a limited number of covers. Your success depends on every guest leaving happy, but you only have one shot to get the service right. Let's break down how you keep the dinner running smoothly, using the same kind of checks Kubernetes uses for its apps.

Before you even open the doors, you need to check the oven is preheated and the gas is on. That is your 'startup probe' — making sure the kitchen itself (your app's container) is alive and ready to start cooking before any orders come in. Once the first diners arrive, you need to constantly check that the fryer is still hot and the chef hasn't burnt the sauce. That is the 'liveness probe' — a heartbeat check that the kitchen is still functioning minute by minute. But even with a working kitchen, you might run out of the evening's special ingredient, like truffle oil. If a waiter tries to serve a dish without it, the guest will be disappointed. So you add a 'readiness probe' to the chalkboard: the special is off the menu until more truffle oil arrives. This tells the wait staff (the Kubernetes service) to stop sending customers to that dish until it can be properly prepared. If the oven does break mid-service, the liveness probe fails, and the head chef pulls the plug on that station completely, letting a new chef step in to get it running again. The guests never know there was a problem — they just get a perfectly timed meal from the stations that are actually ready to serve.

## Core explanation

Kubernetes is a system for running containers (lightweight, self-contained environments for your application code) across a cluster of machines. A cluster is just a group of computers (called nodes) that work together. Your app runs inside a Pod, which is the smallest deployable unit in Kubernetes — a wrapper around one or more containers. The problem is, just because a container is running doesn't mean the application inside it is healthy. It might be stuck in a startup loop, waiting for a database, or running but returning errors. To solve this, Kubernetes lets you define three types of health checks, called 'probes'. A probe is a periodic check that Kubernetes performs to determine the health of your container.

The first type is the startup probe. This is a one-time check that runs when a container first starts. It is designed for applications that take a long time to initialise — for example, a Java application that needs to load huge configuration files or a machine learning model that must be loaded into memory. The startup probe gives the container time to get ready before Kubernetes starts checking its ongoing health. If the startup probe fails, Kubernetes kills the container and restarts it. You configure a startup probe with a period (how often to check), a failure threshold (how many failures are allowed), and a delay (how long to wait before the first check). The key here is that while the startup probe is running, both the liveness and readiness probes are disabled. This prevents them from interfering with a slow-starting application.

The second probe is the liveness probe. Think of this as a 'heartbeat' check. It runs continuously throughout the container's life and answers the question: 'Is the application still running and not stuck in a deadlock?' A deadlock is when part of your code gets frozen waiting for something that will never happen, like a shared resource that another part of your code has locked and forgotten about. If the liveness probe fails, Kubernetes restarts the container automatically. This is crucial because it turns a 'zombie' container — one that is technically running but doing nothing useful — into a fresh, working instance. You can configure a liveness probe to check by making an HTTP GET request to a specific endpoint (like /healthz), or by running a command inside the container (exec probe), or by opening a TCP socket. The standard pattern is to serve a simple health endpoint from your application that returns HTTP 200 when the app is healthy.

The third probe is the readiness probe. This checks if the container is ready to start serving traffic. It runs continuously, just like the liveness probe, but a failure does NOT restart the container. Instead, it removes the Pod from the list of available endpoints that a Kubernetes Service (a way to expose your application to the network) uses for load balancing. This means no new requests are sent to that Pod until the readiness probe succeeds again. This is vital for applications that can temporarily fail — for example, a web server that needs to load a large cache file or an API that relies on an external service that is temporarily down. Without a readiness probe, Kubernetes would keep sending traffic to a broken Pod, causing errors for users.

All three probes share the same configuration options: initialDelaySeconds (wait before first check), periodSeconds (how often to check), timeoutSeconds (how long to wait for a response), successThreshold (how many consecutive successes to consider the probe healthy), and failureThreshold (how many consecutive failures to consider the probe failed). The default values are sensible for many applications, but you must adjust them based on your app's characteristics. For example, for a slow-starting app, you would increase initialDelaySeconds or use a startup probe. For a really slow starter, you might need both a startup probe with a high failure threshold and a liveness probe with a generous timeout.

Why does this matter for your CKAD exam? Because the exam will ask you to write YAML (a human-readable data-serialisation language used to define Kubernetes resources) that configures these probes correctly. You might be given a scenario where an application is crashing in production, and you need to add appropriate probes to make it self-healing. Or you might be asked to fix a probe that is too aggressive — restarting a container incorrectly because the probe timeout is too short. Understanding the precise behaviour of each probe type is essential. The official Kubernetes documentation is your best friend here, but the CKAD exam is open-book, so you can look up the exact syntax as long as you know what to search for.

A final important detail: probes are defined in the container spec of a Pod. Each container in a Pod can have its own set of probes. This means you can have a sidecar container (a helper container that runs alongside your main app) with different health checks than the main application. For example, a reverse proxy container might have a readiness probe that checks its own configuration, independent of the main app container's health. This modularity is one of Kubernetes' strengths—you can monitor and recover from failures at a granular level.

## Real-world context

Let us imagine you work for a company called 'QuickServe', which runs an online food ordering platform. The platform is built using microservices — small, independent applications that each handle one part of the business, like user accounts, order management, and payment processing. Each service runs in its own container, orchestrated by Kubernetes. Your team is responsible for the 'order-service', which processes incoming orders and talks to the payment service. Without health checks, if the order-service crashes, customers would submit orders that disappear into a void. With proper probes, the system recovers automatically.

Here is what an IT professional at QuickServe does step by step:

- The developer writes the code for the order-service and includes a simple HTTP endpoint at `/healthz` that returns a 200 status code and a message like 'OK'. This endpoint is not a fancy API endpoint — it is a dedicated endpoint that does nothing but report health.
- The developer creates the Kubernetes Deployment YAML (a definition file that tells Kubernetes how to manage the order-service). Inside the container spec, they add a liveness probe that hits `GET /healthz` every 10 seconds, with a 5-second timeout and a failure threshold of 3. This means the pod will be restarted if the endpoint fails to respond three times in a row.
- The order-service also needs to load a large menu database when it starts up, which takes about 30 seconds. Without a startup probe, the liveness probe would start checking immediately and might restart the container prematurely. So the developer adds a startup probe with an initial delay of 5 seconds and a failure threshold of 10, giving the container up to 50 seconds to start.
- During the day, the payment service occasionally has a brief outage of a few seconds. When this happens, the order-service correctly returns an error for that specific request, but the liveness probe should not kill the container for a transient problem. Instead, the developer adds a readiness probe that checks a different endpoint: `/ready`, which verifies that the connection to the payment service is working. If the payment service is down, the readiness probe fails, and Kubernetes stops sending new orders to this Pod. The existing Pod continues running, and the liveness probe (checked separately) stays healthy because the order-service code itself is not stuck.
- Once the payment service recovers, the readiness probe succeeds again, and Kubernetes automatically adds the Pod back into the load balancing pool. Customers experience a slight delay in their order confirmation, but no orders are lost.
- The IT professional also monitors the probes in production using Kubernetes dashboards and logs. They see metrics like 'probe success rate' and 'restart count'. If the restart count spikes, they know the liveness probe is too sensitive and needs tweaking — for example, increasing the timeout from 5 to 10 seconds.

The real world value is clear: these probes save the team from being woken up at 2 AM by a pager alert. Instead of manually restarting containers or debugging why users are seeing errors, the system heals itself. For the CKAD exam, you are expected to know the exact YAML syntax for these probes, the differences between them, and the scenarios where you would choose one over the other. A common task in the exam is to look at a flawed Deployment manifest (the YAML file) and identify which probe is configured incorrectly — for example, a liveness probe that uses a command that never returns a non-zero exit code, or a readiness probe that points to a page that returns a 500 error even when the app is healthy.

## Exam focus

The CKAD exam tests your ability to write and modify Kubernetes manifests, specifically YAML, to configure health checks correctly. You will not be asked to explain theory in an essay — you will be given an existing YAML file with a problem, and you must fix it by adding or adjusting probes. The exam is performance-based, meaning you are given a real cluster to work on, and you must use `kubectl` commands (the command-line tool for interacting with Kubernetes) to complete tasks.

Here are the exact topics they love to test:

- The difference between liveness and readiness probes: This is the most common trap. The exam will give you a scenario like 'An application is temporarily over capacity and slow to respond, but it is not dead.' If you use a liveness probe, Kubernetes will restart it — which is wrong because the app is still functional. You need a readiness probe to take it out of service temporarily. Traps often include using 'liveness' when 'readiness' is needed, or vice versa.
- Startup probe usage: They love to test scenarios where an application takes more than 60 seconds to start. Without a startup probe, the liveness probe will kill it before it finishes booting. You must know that setting `initialDelaySeconds` on a liveness probe is not enough for very slow apps — you need a dedicated startup probe with a higher `failureThreshold`.
- Probe configuration fields: You must memorise the exact meaning of `initialDelaySeconds`, `periodSeconds`, `timeoutSeconds`, `successThreshold`, and `failureThreshold`. A typical question might ask you to set a probe that checks every 15 seconds, with a 5-second timeout, and restarts after 3 failures. You need to write the exact YAML.
- HTTP, TCP, and Exec probes: The exam expects you to know the syntax for each type. For HTTP probes, you specify the `path` and `port`. For TCP probes, you specify the `port`. For Exec probes, you specify a `command` (array of strings). They might give you a container that runs a health check script and ask you to use an exec probe.
- The effect of probe failures on Pod lifecycle: You need to understand that a liveness probe failure leads to a container restart, while a readiness probe failure leads to removal from the Service endpoint list. A startup probe failure also restarts the container, but only during the initial startup period.
- YAML syntax errors: Since the exam uses a real terminal, common mistakes include indentation errors (YAML is extremely sensitive to spaces) or misplacing the probe block inside `spec.containers[].livenessProbe` versus `spec.containers[].readinessProbe`.

To practise, you should use the official Kubernetes documentation's 'Configure Liveness, Readiness and Startup Probes' page. During the exam, you can open this page in a browser. However, do not rely on copying code verbatim — you need to understand the logic so you can adapt the examples to the specific scenario.

Common traps include:
- Using a readiness probe when the app is actually dead and needs a restart — this leads to a 'zombie' app that never recovers.
- Setting `initialDelaySeconds` too low, causing the probe to fail because the app hasn't started yet.
- Using an HTTP probe with a path that returns a 200 OK even when the app is broken (e.g., a static page that always loads).
- Forgetting that the probe is defined at the container level, not the Pod level.

Memorise these definitions for the exam:
- Startup Probe: checks if the container has started. Disables liveness and readiness until it succeeds.
- Liveness Probe: checks if the container is still running. Restarts on failure.
- Readiness Probe: checks if the container is ready to serve traffic. Removes from service on failure.
- Probe Syntax: use `httpGet`, `tcpSocket`, or `exec` as the probe handler type.

## Step by step

1. **Identify the Application's Needs** — Before writing a probe, you must understand your application's behaviour. Does it start slowly? Does it have temporary dependencies that may fail? Does it handle traffic immediately? This determines which probes you need and their parameters.
2. **Add a Startup Probe for Slow-Starting Apps** — If your container takes more than 60 seconds to start (e.g., loading a large model or connecting to a database), add a startup probe with a high failureThreshold (e.g., 30) and a short period (e.g., 5 seconds) to give it up to 150 seconds to start. This disables liveness and readiness during startup.
3. **Configure a Liveness Probe for Self-Healing** — Add a liveness probe to detect deadlocks or process freezes. Use an HTTP endpoint like /healthz that returns 200 if the app's main loop is responsive. Set periodSeconds to 10 and failureThreshold to 3 so the container restarts within 30 seconds of becoming unresponsive.
4. **Add a Readiness Probe for Traffic Management** — Configure a readiness probe to check external dependencies (e.g., reachability of a database or cache). Use a separate endpoint like /ready that returns 200 only when all dependencies are available. When this probe fails, traffic stops until it recovers.
5. **Test with kubectl and Monitor Logs** — After writing the YAML, deploy it using `kubectl apply -f deployment.yaml`. Use `kubectl describe pod <name>` to see probe events and `kubectl logs <pod>` to debug failures. Adjust timeoutSeconds or failureThreshold if the probe is too aggressive or too lenient.

## Comparisons

### Liveness Probe vs Readiness Probe

**Liveness Probe:**
- Restarts the container on failure
- Checks if the app is alive and not deadlocked
- Used for self-healing from process crashes

**Readiness Probe:**
- Only removes Pod from Service endpoints on failure
- Checks if the app is ready to serve traffic
- Used for graceful handling of temporary issues

### Startup Probe vs Liveness Probe

**Startup Probe:**
- Runs only during initial startup
- Disables liveness and readiness until it passes
- Has a higher failure threshold to accommodate slow starts

**Liveness Probe:**
- Runs continuously throughout the container's life
- Does not affect other probes' operation
- Has a lower failure threshold for fast failure detection

### HTTP Probe vs Exec Probe

**HTTP Probe:**
- Makes an HTTP GET request to a specified endpoint
- Requires the application to have an HTTP server running
- Considers HTTP status 200-399 as healthy

**Exec Probe:**
- Runs a shell command inside the container
- Works for applications without HTTP servers
- Considers exit code 0 as healthy, any other as failure

## Diagram

_This flowchart shows the lifecycle of a Pod with probes: startup probe controls initialisation, liveness probe triggers restarts on failure, and readiness probe manages traffic routing._

```mermaid
flowchart TD
    A[Pod Created] --> B{Startup Probe Defined?}
    B -- Yes --> C[Startup Probe Runs]
    C --> D{Startup Probe Passes?}
    D -- No --> E[Container Restarted]
    E --> B
    D -- Yes --> F[Liveness + Readiness Probes Active]
    B -- No --> F
    F --> G{Liveness Probe Passes?}
    G -- No --> E
    G -- Yes --> H{Readiness Probe Passes?}
    H -- No --> I[Pod Removed from Service]
    H -- Yes --> J[Pod Receives Traffic]
    I --> F
    J --> F
```

## Common misconceptions

- **Misconception:** If a liveness probe fails, Kubernetes deletes the Pod and creates a new one. **Reality:** A liveness probe failure causes the container to be restarted, not the Pod itself. The Pod remains, but its container gets recreated. The Pod name and IP address usually change, but the Deployment controller ensures a new Pod is running if needed. (Beginners confuse container restarts with Pod restarts. In Kubernetes, Pods are the smallest deployable unit, but a single Pod can have multiple containers. The liveness probe operates at the container level, not the Pod level.)
- **Misconception:** You should always use an HTTP probe because it is the most reliable. **Reality:** HTTP probes are great for web applications that expose an HTTP endpoint, but they are not suitable for all applications. For example, a background worker that does not listen on any port cannot respond to HTTP probes — you would need an exec probe that runs a custom health check script. The choice depends on the application type. (Newcomers think HTTP is universal, but many applications (like batch job processors, message queue consumers, or CLI tools) do not have an HTTP server. The exam will test your ability to choose the correct probe type based on the container's functionality.)
- **Misconception:** Once a readiness probe fails, the container is killed and restarted like liveness. **Reality:** A readiness probe failure does NOT restart the container. It only removes the Pod from the list of endpoints that a Service sends traffic to. The container continues running, and if the readiness probe later succeeds, the Pod is added back automatically. (The different consequences of failure in liveness vs readiness are confusing. Beginners memorise that both 'fail' but do not internalise the distinct actions taken by Kubernetes. This is a top exam trap.)
- **Misconception:** Startup probes are optional and you can achieve the same effect by increasing initialDelaySeconds on the liveness probe. **Reality:** No, increasing initialDelaySeconds only delays the start of the liveness probe. Once it starts, it checks health immediately. If the application takes a long time to initialise, the liveness probe will still fail and restart it. A startup probe gives the application a longer window to become healthy without interference from the liveness probe, and it disables liveness/readiness checks until it passes. (This misconception arises because both delay approaches seem similar. The key difference is that a startup probe actively monitors progress, while initialDelaySeconds is a static wait. The exam loves this distinction.)
- **Misconception:** A probe can only check one type of health — either liveness, readiness, or startup — per container. **Reality:** You can define all three probes on the same container simultaneously. They serve different purposes and can coexist. For example, a container can have a startup probe for slow initialisation, a liveness probe for ongoing heartbeat checking, and a readiness probe for checking dependencies like database connectivity. (New learners think probes are mutually exclusive. In reality, combining them gives fine-grained control over different phases of a container's lifecycle.)

## Key takeaways

- A startup probe runs only once when the container first starts, giving slow-to-initialise applications time to become healthy before liveness or readiness checks begin.
- A liveness probe failure restarts the container, while a readiness probe failure only removes the Pod from the Service endpoint list until it recovers.
- All three probes can be defined on the same container without conflict, each serving a distinct purpose in managing the container's lifecycle.
- HTTP probes must be configured with a path that returns HTTP 200-399 to be considered healthy; a 5xx response counts as a failure.
- Exec probes run a command inside the container and consider it healthy if the command exits with code 0, and unhealthy for any non-zero exit code.
- The default failure threshold for probes is 3 consecutive failures, which you can adjust for applications with unpredictable behaviour.

## FAQ

**Can I use the same endpoint for both liveness and readiness probes?**

Yes, you can use the same HTTP endpoint for both, but it is not recommended because they serve different purposes. A liveness endpoint should only check if the application process is alive, while a readiness endpoint should also verify external dependencies. Using the same endpoint could mask readiness issues.

**What happens if I don't define any probes?**

Without probes, Kubernetes has no way to know if your container is healthy. It will keep sending traffic to a broken or frozen container, causing errors for users. You rely on manual intervention or external monitoring to detect failures.

**What is the difference between initialDelaySeconds and a startup probe?**

initialDelaySeconds is a static wait before starting the first check of any probe. A startup probe is a separate probe that runs during the entire startup phase and disables liveness/readiness until it succeeds. A startup probe is more flexible because it actively checks progress, while initialDelaySeconds just pauses.

**Can a readiness probe be used to restart a container?**

No, readiness probe failures never restart the container. They only affect whether the Pod receives traffic. If you need a container to restart on a specific condition, you must use a liveness probe (or combine both).

**How do I check if my probe is working correctly?**

Use `kubectl describe pod <pod-name>` and look for the 'Events' section. You will see messages like 'Liveness probe failed: HTTP probe failed with statuscode: 500' or 'Readiness probe succeeded'. This tells you exactly what the probe sees.

**What is a good default timeout for an HTTP probe?**

A timeout of 5 seconds is reasonable for most HTTP probes. If your application is slow due to heavy load, you might increase it to 10 or 15 seconds. Setting it too low causes false failures; setting it too high delays failure detection.

## Check your understanding

1. **What happens to a Pod when its readiness probe fails?**

   Answer: The Pod is removed from the list of endpoints that the Service uses to route traffic, but the container is not restarted. Traffic resumes automatically when the readiness probe succeeds again.

2. **Which probe should you use for an application that takes 120 seconds to start but then runs normally?**

   Answer: You should use a startup probe with a failureThreshold high enough to allow 120 seconds for startup (e.g., periodSeconds=10, failureThreshold=12). This prevents the liveness probe from prematurely restarting the container.

3. **Is it possible to have both liveness and readiness probes on the same container?**

   Answer: Yes, you can and often should define both. The liveness probe ensures the container restarts if it becomes unresponsive, while the readiness probe handles temporary issues like dependency outages.

4. **What is the default failure threshold for a probe if not specified?**

   Answer: The default failure threshold is 3. This means the probe is considered failed after three consecutive failures, leading to a container restart (for liveness) or traffic removal (for readiness).

5. **What is the key difference between using initialDelaySeconds and a startup probe?**

   Answer: initialDelaySeconds is a static wait before starting the first probe check, while a startup probe actively checks startup progress and disables liveness/readiness until it succeeds. A startup probe is more flexible for slow-starting applications.

---

Interactive version with quiz and diagrams: https://courseiva.com/learn/cncf-ckad/readiness-and-liveness-probes