Cluster Hardening: Node and Container Security

Node and container security is a set of practices to protect the individual computers (nodes) that run your applications (containers) inside a Kubernetes cluster, and to protect the applications themselves from harming each other or the underlying system. A Kubernetes cluster is a group of computers, called nodes, that work together. Each node can be a physical machine or a virtual machine (a simulated computer running inside another computer). On each node, the container runtime (like Docker or containerd) runs your application inside an isolated environment called a container. A container is a lightweight, standalone, executable package that includes everything needed to run a piece of software.

Let's start with node hardening. A node that is not hardened is like a house with all the doors unlocked. Hardening means making it difficult for an attacker to compromise that specific computer. This begins with the host operating system (OS), the fundamental software that makes the computer run. You must regularly apply security patches (updates that fix known vulnerabilities) to the OS. You should disable any services that are not required. A service is a program that runs in the background, like a file-sharing service or a web server. If you don't need it, turn it off. You should also configure authentication (the process of verifying who you are) and authorisation (the process of deciding what you are allowed to do) for accessing the node itself, typically by using SSH (Secure Shell) keys instead of passwords. A common tool is AppArmor (Application Armor), a Linux kernel security module that restricts what programs can do. You can also use seccomp (Secure Computing Mode), which limits the system calls a container can make. A system call is a request the program makes to the operating system kernel (the core of the OS) to do something, like open a file or connect to a network. By limiting these calls, you make it much harder for a compromised container to attack the host node.

The goal of node hardening is to reduce what security experts call the 'attack surface' — the total number of ways an attacker could potentially try to break into the system.

Now, let's look at container security. A container should be run with only the minimum necessary permissions. This is the 'principle of least privilege'. A container doesn't need root access (administrator power) to run your web application. If a container runs as root and is compromised, the attacker instantly has full control over that container and has an easier path to escaping onto the host node. You should use a dedicated user for the container, defined in the container image (the template used to build the container). You should also make the container's filesystem read-only whenever possible. A filesystem is the structure where files are stored. Making it read-only means the container cannot write new files or modify existing ones, which stops malware from saving itself onto the disk.

Kubernetes provides Pod Security Standards (PSS) to help. A pod is the smallest deployable unit in Kubernetes, which can hold one or more containers. Pod Security Standards define three policies:

These standards replace older concepts like Pod Security Policies (PSP) which were deprecated (removed from newer versions of Kubernetes). You will apply these standards using Admission Controllers. An Admission Controller is a piece of code that intercepts requests to the Kubernetes API (the main control interface) to create a pod, and either allows or denies the request based on the security rules you have set.

Why does this matter? In older models, you trusted the container and the system. Now, you trust nothing. You build layers of security so that even if one layer fails, another layer catches the attacker. For the CKS exam, the focus is on practical tasks: editing configuration files, checking running containers, and applying security contexts (settings inside a pod's definition file that specify user IDs, filesystem permissions, and other restrictions). You will also need to use tools like kube-bench (a tool that checks your cluster against security best practices) and kube-hunter (a tool that looks for vulnerabilities in your cluster). You might also need to use Falco, a security monitoring tool that watches for suspicious behaviour in your containers and nodes. This is not just theory; you will be expected to type commands into a terminal to harden a cluster in a time-pressured exam environment.

An IT professional working as a Kubernetes administrator at a mid-sized e-commerce company is tasked with hardening the cluster before a major sales event. They know that during high traffic, attempts to exploit vulnerabilities increase. Here is what they actually do, step by step.

First, they audit the nodes. They log into each machine and run kube-bench. This tool automatically checks the node against the CIS (Center for Internet Security) Kubernetes Benchmark, a published set of security recommendations. The report shows that SSH password authentication is still enabled on two nodes, and the kubelet service (the primary agent that runs on each node) is listening on an insecure port. The administrator disables password authentication, forces the use of SSH keys only, and reconfigures the kubelet to use only secure, encrypted ports (port 10250 instead of 10255).

Next, they analyse existing containers. They use the command 'kubectl get pods --all-namespaces' to see every running pod. Then they inspect the security context of a critical payment processing pod. They output the pod's definition file and see that the container is set to 'privileged: true'. This is a major risk. A privileged container has all the same access to the host as the root user. The administrator works with the development team to tweak the container code. They remove the privileged flag, and instead set 'allowPrivilegeEscalation: false' and 'runAsUser: 1000'. They drop all Linux capabilities (specific permissions a container requests, like the ability to change network settings) except those absolutely needed. This means the container runs as a non-root user with minimal permissions.

Then, they implement Pod Security Standards. The administrator creates a namespace for the payment services called 'payments'. They add a label to that namespace: 'pod-security.kubernetes.io/enforce: restricted'. This label tells Kubernetes to reject any pod that does not meet the 'restricted' policy level. The administrator then tries to deploy the old unhardened pod definition. The deployment fails because the container is running as root. The administrator works with the developer to fix the image to run as a non-root user, and after the fix, the pod is created successfully.

Finally, they set up runtime security. They install Falco via a Kubernetes DaemonSet (a controller that ensures a copy of a pod runs on every node). Falco will now monitor system calls and alert the operations team if a container tries to spawn a shell or read sensitive files like '/etc/shadow'. This gives the team real-time visibility into attacks that bypass the initial hardenings. The team also configures Falco to send alerts to a central logging system (like the ELK stack — Elasticsearch, Logstash, Kibana) so they can investigate incidents after the fact.

This real-world workflow ensures that even if a developer accidentally pushes a vulnerable container, the cluster's built-in defences block it. The administrator does not just trust the people using the system; they design the system so that mistakes are caught automatically.