This chapter covers container and Kubernetes security analysis, a critical topic for the CS0-003 exam. Containers and Kubernetes have revolutionized application deployment but introduced new attack surfaces that security analysts must understand. Expect 10-15% of exam questions to touch on container security, including image vulnerabilities, runtime threats, Kubernetes RBAC misconfigurations, and network policy enforcement. We will explore the underlying mechanisms, common misconfigurations, and how to audit and secure these environments using industry-standard tools and practices.
Jump to a section
Imagine a modern shipping port. Each shipping container is a standardized, self-contained unit that carries cargo (your application code and dependencies). The container has its own walls (namespace isolation), its own manifest (Dockerfile or image), and is designed to be loaded onto any ship, truck, or train without modification. Kubernetes is the harbor master and the entire port infrastructure. The harbor master (Kubernetes control plane) receives orders (desired state) and assigns cranes (kubelets) to move containers onto specific docking spots (pods). The port has scheduling rules (scheduler) to decide which berth gets which container based on weight, destination, and priority. The harbor also has a registry of all containers (image registry), a network of roads and cranes (container network interface, or CNI), and storage warehouses (persistent volumes). Security at this port is critical: you must verify the identity of every container (image signing), inspect cargo for contraband (vulnerability scanning), control who can operate cranes (RBAC), and monitor for unauthorized containers sneaking in (admission controllers). Just as a port can be crippled by a single mislabeled container or a crane malfunction, a Kubernetes cluster can be compromised by a single vulnerable image or misconfigured RBAC policy. The analogy holds because both systems are about standardized, isolated units managed by a powerful orchestrator, and security must be layered at every level: the container itself, the runtime, the network, and the orchestrator.
What Are Containers and Why Do They Matter for Security?
Containers are lightweight, portable units that package an application and its dependencies together. They leverage operating system-level virtualization, sharing the host OS kernel but isolating the application's view of the filesystem, network, process tree, and more via Linux namespaces and control groups (cgroups). From a security perspective, containers are not as isolated as virtual machines (VMs). A VM runs a separate kernel, so a kernel exploit in the VM does not directly compromise the host. In contrast, a container shares the host kernel; a kernel exploit can escape the container and gain host access. This fundamental difference is why container security requires additional layers: image scanning, runtime security, and least-privilege configurations.
How Container Isolation Works: Namespaces and Cgroups
Linux namespaces partition kernel resources so that each container sees its own view of: - PID namespace: Container processes have their own process tree, starting at PID 1. - Network namespace: Each container gets its own network stack (interfaces, IP addresses, routing tables). - Mount namespace: Filesystem mounts are isolated. - UTS namespace: Hostname and domain name are isolated. - IPC namespace: Inter-process communication resources (e.g., System V IPC, POSIX message queues) are isolated. - User namespace: User and group IDs can be mapped, so root inside the container is not root outside.
Control groups (cgroups) limit and account for resource usage (CPU, memory, disk I/O). They prevent a container from exhausting host resources and causing denial of service.
Container Images and Vulnerability Scanning
A container image is a read-only template with layers, each layer being a set of filesystem changes. Images are stored in registries (e.g., Docker Hub, Amazon ECR, Google Container Registry). Security starts with the image: scanning for known vulnerabilities (CVEs) in the base OS packages and application dependencies. Tools like Trivy, Clair, and Anchore Grype compare package versions against vulnerability databases (e.g., NVD, Red Hat OVAL). The exam expects you to know that image scanning should be integrated into the CI/CD pipeline to catch vulnerabilities before deployment. Also, images should be signed (e.g., using Docker Content Trust or Notary) to ensure integrity and authenticity.
Kubernetes Architecture and Security Boundaries
Kubernetes (K8s) orchestrates container deployment, scaling, and management. Its architecture consists of: - Control plane: API server, etcd (key-value store), scheduler, controller manager. - Nodes: Worker machines running kubelet, kube-proxy, and container runtime.
Security domains: - Cluster-level security: Authentication, authorization (RBAC), admission control, network policies, secrets management. - Node-level security: Host OS hardening, kubelet security, container runtime security. - Workload-level security: Pod security standards (formerly PodSecurityPolicies), seccomp, AppArmor, and resource limits.
Authentication and Authorization in Kubernetes
- Authentication: Users (human or service accounts) authenticate via client certificates, bearer tokens (static, service account, or OIDC), or proxy credentials. The API server validates the identity. - Authorization: Determines what an authenticated user can do. Kubernetes supports: - Node authorization: For kubelets. - ABAC (Attribute-Based Access Control): Static policy file. - RBAC (Role-Based Access Control): Most common. Roles define permissions on resources (verbs like get, list, create, delete). RoleBindings assign roles to users/groups/service accounts. - Webhook: External authorization service. - AlwaysAllow/AlwaysDeny: Simple modes.
The exam focuses on RBAC. A common misconfiguration is granting cluster-admin privileges to a service account that only needs read access to a single namespace. Use the principle of least privilege.
Admission Controllers
Admission controllers intercept requests to the API server after authentication and authorization but before object persistence. They can validate, mutate, or reject requests. Built-in admission controllers include: - AlwaysPullImages: Ensures images are always pulled, preventing use of locally cached images that might be stale or malicious. - SecurityContextDeny: Rejects pods that request privileged security contexts (e.g., privileged containers, hostNetwork). - PodSecurityPolicy (PSP): Deprecated in 1.21, removed in 1.25. Replaced by Pod Security Standards (PSS) with three policies: Privileged, Baseline, Restricted. - ValidatingAdmissionWebhook and MutatingAdmissionWebhook: Allow custom policies.
The exam expects you to know that admission controllers are critical for enforcing security policies at the cluster level. For example, a validating webhook can enforce that all images come from an approved registry.
Network Policies
By default, all pods can communicate with each other. Network policies implement micro-segmentation using label selectors and namespace selectors. They are implemented by a CNI plugin that supports them (e.g., Calico, Cilium, Weave Net). A NetworkPolicy is a namespaced resource that specifies: - podSelector: Selects pods to which the policy applies. - policyTypes: Ingress, Egress, or both. - ingress rules: Allowed sources (ipBlock, namespaceSelector, podSelector) and ports. - egress rules: Allowed destinations and ports.
Example policy that allows only frontend pods to talk to backend pods on TCP 8080:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: backend-allow-frontend
spec:
podSelector:
matchLabels:
app: backend
policyTypes:
- Ingress
ingress:
- from:
- podSelector:
matchLabels:
app: frontend
ports:
- protocol: TCP
port: 8080A common exam trap: forgetting that NetworkPolicy is namespaced and does not affect pods in other namespaces unless namespaceSelector is used. Also, if no NetworkPolicy is applied, all traffic is allowed; if any policy selects a pod, only traffic explicitly allowed is permitted.
Secrets Management
Kubernetes Secrets are base64-encoded objects intended for sensitive data like passwords, tokens, and keys. However, base64 is not encryption; anyone with access to etcd or the API can decode them. For real security, use encryption at rest for etcd, and consider external secrets management (e.g., HashiCorp Vault, AWS Secrets Manager) with CSI drivers. The exam expects you to know that Secrets are not encrypted by default and that RBAC should restrict access to Secrets.
Runtime Security and Auditing
Runtime security monitors container behavior for anomalies. Tools like Falco (from Sysdig) use kernel modules or eBPF to capture system calls and compare against rules. For example, a rule might alert when a container spawns a shell or reads /etc/shadow. The exam may ask about Falco rules syntax or common triggers.
Audit logging in Kubernetes records API requests. The audit policy defines what events are logged (e.g., RequestResponse, Metadata). Audit logs are essential for forensics.
Common Attack Vectors
Vulnerable images: Using base images with known CVEs.
Misconfigured RBAC: Overly permissive roles, especially cluster-admin.
Privileged containers: Running containers with privileged: true or hostPID: true can escape.
Exposed dashboards: Unauthenticated access to Kubernetes dashboard.
Insecure kubelet: kubelet API exposed without authentication (port 10250).
Supply chain attacks: Compromised images from public registries.
Network segmentation bypass: Missing network policies allow lateral movement.
Tools and Commands for Security Analysis
- kube-bench: Checks Kubernetes against CIS benchmarks. - kube-hunter: Penetration testing tool for Kubernetes. - kubectl auth can-i: Check if a user can perform an action.
kubectl auth can-i create pods --as=system:serviceaccount:default:my-sakubectl describe pod: Shows security context, volumes, etc.
kubectl get networkpolicy: List network policies.
kubectl get rolebindings --all-namespaces: List all RBAC bindings.
kubectl logs <pod>: Container logs.
crictl: CLI for CRI-compatible runtimes (containerd, CRI-O).
docker inspect: Shows container configuration, including capabilities and mounts.
CIS Benchmarks for Kubernetes and Containers
The Center for Internet Security publishes benchmarks for: - Docker: Covers host configuration, daemon configuration, etc. - Kubernetes: Covers control plane, nodes, etcd, RBAC, etc. - Container images: Minimize host OS footprint.
Key recommendations include: use Linux security modules (AppArmor, SELinux), enable seccomp, restrict container capabilities, and run containers as non-root.
Image Scanning and Signing
Before deployment, scan container images for vulnerabilities using tools like Trivy or Clair. Integrate scanning into the CI/CD pipeline; reject images with critical or high CVEs. Also, sign images using Docker Content Trust (DCT) or Notary to ensure integrity. The registry should enforce signature verification. This step prevents known vulnerabilities from entering the cluster. A common mistake is scanning only at build time; images should also be rescanned periodically as new CVEs are discovered.
Configure RBAC with Least Privilege
Define Roles and ClusterRoles with minimal permissions. Use RoleBindings and ClusterRoleBindings to assign roles to users, groups, or service accounts. Avoid using cluster-admin unless absolutely necessary. For example, a service account for a monitoring tool should only have get and list permissions on pods and nodes in its namespace. Regularly audit bindings with `kubectl get rolebindings --all-namespaces`. A common trap is granting create permissions on pods, which can allow privilege escalation via mounting host paths.
Apply Pod Security Standards
Use Pod Security Admission (PSA) to enforce Pod Security Standards at the namespace level. Choose between Privileged, Baseline, or Restricted policies. For example, the Restricted policy disallows privileged containers, host networking, and host PID. Configure via labels on namespaces: `pod-security.kubernetes.io/enforce=restricted`. This replaces deprecated PodSecurityPolicies. Test with warnings before enforcing.
Implement Network Policies
Create NetworkPolicy resources to segment traffic between pods. Start by denying all ingress and egress (a 'default deny' policy), then allow specific flows. For example, allow frontend pods to access backend pods on port 8080. Ensure the CNI plugin supports NetworkPolicy (e.g., Calico, Cilium). Test policies using `kubectl run` with netcat or similar. A common mistake is forgetting that NetworkPolicy is namespaced; cross-namespace traffic requires namespaceSelector.
Enable Audit Logging and Runtime Monitoring
Configure the Kubernetes API server with an audit policy that logs all requests at the RequestResponse level for sensitive resources (e.g., Secrets, ConfigMaps). Send logs to a central SIEM. Deploy runtime security tools like Falco to monitor container behavior. For example, Falco can alert when a container runs a shell. Regularly review audit logs for unauthorized API calls. Ensure audit log retention meets compliance requirements.
Enterprise Scenario 1: Financial Services Microservices Platform
A large bank deploys hundreds of microservices on a Kubernetes cluster to process transactions. Security is paramount due to regulatory requirements (PCI-DSS, SOX). The bank uses image scanning with Trivy in the CI/CD pipeline, rejecting any image with a CVE score above 7.0. They enforce Pod Security Standards with the 'Restricted' policy in production namespaces, preventing privileged containers. Network policies isolate each microservice: only the API gateway can talk to the frontend, and only the frontend can talk to the backend. RBAC is tightly controlled: each team has a dedicated namespace with a Role that allows only the necessary operations (e.g., developers can create pods but not delete them). Audit logs are shipped to a SIEM (Splunk) and monitored for anomalies. Common issues: developers accidentally granting cluster-admin for convenience, which is caught by periodic audits using kube-bench.
Enterprise Scenario 2: E-commerce Platform with Public Registry Images
An e-commerce company uses a mix of public and private images. They experienced a supply chain attack when a popular Node.js image from Docker Hub contained a cryptominer. After the incident, they implemented image signing with Notary and configured admission controllers to only allow images signed by their internal registry. They also use Falco to detect runtime anomalies, such as unexpected outbound network connections. They run periodic kube-hunter scans to identify misconfigurations like an exposed kubelet API. The biggest challenge is balancing security with developer velocity; they use a 'break glass' process for urgent deployments that bypass some checks, but those are logged and reviewed.
Performance and Scale Considerations
At scale (thousands of pods), network policies can become complex and impact performance if not designed carefully. CNI plugins like Cilium use eBPF for efficient policy enforcement. Audit logging can generate massive volumes; tune the audit policy to log only necessary events. RBAC with too many roles can become unmanageable; use aggregated ClusterRoles and role templates. Image scanning should be done in parallel to avoid pipeline delays.
What CS0-003 Tests on Container and Kubernetes Security (Objective 1.4)
The exam focuses on your ability to analyze security issues in containerized environments. Specific objectives include:
- 1.4 Given a scenario, analyze the output from a security control. You may be given output from kubectl describe pod, kubectl get networkpolicy, or a Falco alert and asked to identify vulnerabilities or misconfigurations.
- 1.5 Given a scenario, analyze the output from common tools. Tools like kube-bench, kube-hunter, and Trivy output may appear.
Common Wrong Answers and Why Candidates Choose Them
"PodSecurityPolicy is the recommended way to enforce pod security." This is wrong because PSP is deprecated and removed in Kubernetes 1.25. The correct answer is Pod Security Admission (PSA) with Pod Security Standards. Candidates who studied older material will choose PSP.
"Secrets are encrypted by default." Wrong. Secrets are only base64-encoded. Candidates confuse encoding with encryption. The correct answer: Secrets should be encrypted at rest using etcd encryption or external secrets management.
"NetworkPolicy allows all traffic by default." Wrong. The default behavior when no policy exists is to allow all traffic. But once a policy selects a pod, only traffic explicitly allowed is permitted. Candidates often forget the 'default allow' vs 'default deny' nuance.
"Containers provide the same isolation as virtual machines." Wrong. Containers share the host kernel, so isolation is weaker. A kernel exploit can escape a container. Candidates may confuse container isolation with VM isolation.
Specific Numbers, Values, and Terms That Appear on the Exam
CIS Benchmark: Know that kube-bench checks against CIS benchmarks for Kubernetes and Docker.
Falco rules: Falco uses syscalls; common rules include "Terminal shell in container" (spawns a shell).
Trivy: Scans for CVEs in OS packages and application dependencies.
Default deny policy: A NetworkPolicy that selects all pods but has no rules effectively denies all traffic.
kubectl auth can-i: Command to check permissions.
Admission controllers: AlwaysPullImages, SecurityContextDeny, PodSecurityPolicy (deprecated), ValidatingAdmissionWebhook.
Namespaces: Not a security boundary; RBAC and NetworkPolicy are needed.
Edge Cases and Exceptions
HostNetwork pods: Bypass network policies because they use the host's network stack. If a pod has hostNetwork: true, NetworkPolicy does not apply to it.
Static pods: Created by the kubelet directly without going through the API server; most security controls (RBAC, admission controllers) do not apply.
Init containers: Run before app containers; they can have different security contexts. Ensure they are also scanned and restricted.
Service account tokens: Mounted into pods by default; if a pod is compromised, the attacker can use the token to access the API. Use bound service account tokens (projected volumes) and disable automount.
How to Eliminate Wrong Answers Using the Underlying Mechanism
When faced with a question about container escape, think about the shared kernel. Any answer that suggests containers are as secure as VMs is wrong. For RBAC questions, remember that a RoleBinding grants permissions within a namespace; a ClusterRoleBinding grants cluster-wide permissions. If a service account needs access to multiple namespaces, you need a ClusterRole and ClusterRoleBinding, or multiple RoleBindings. For network policy questions, remember that if no policy selects the pod, all traffic is allowed. If a policy selects the pod, only allowed traffic is permitted. This is a frequent source of confusion.
Containers share the host kernel; isolation is weaker than VMs.
Always scan container images for vulnerabilities before deployment.
Use Pod Security Standards (Restricted) to enforce least-privilege pod configurations.
Kubernetes Secrets are base64-encoded, not encrypted; use encryption at rest and external secrets management.
Network policies are namespaced and default allow; create a default deny policy first.
RBAC should follow least privilege; avoid cluster-admin grants.
Admission controllers like AlwaysPullImages and webhooks enforce security policies at API request time.
Audit logging and runtime monitoring (Falco) are essential for detecting attacks.
CIS benchmarks provide hardening guidelines for Docker and Kubernetes.
kubectl auth can-i is the command to verify permissions.
These come up on the exam all the time. Here's how to tell them apart.
Docker
Single-host container runtime; manages containers on one machine.
Uses Docker daemon; security concerns include daemon socket exposure.
Networking: bridge, host, overlay modes; no built-in network policies.
Compose for multi-container apps, but no cluster-level orchestration.
Security scanning via Docker Scan or third-party tools integrated with Docker Hub.
Kubernetes
Multi-host orchestrator; manages containers across a cluster.
Uses container runtime interface (CRI); can use containerd, CRI-O, or Docker.
Has NetworkPolicy for micro-segmentation; requires CNI plugin.
Built-in RBAC, admission controllers, and secrets management.
Tools like kube-bench, kube-hunter, and Falco for security auditing.
Mistake
Containers provide the same level of isolation as virtual machines.
Correct
Containers share the host kernel and use namespaces and cgroups for isolation, which is weaker than hypervisor-based VM isolation. A kernel exploit can allow a container to access the host.
Mistake
Kubernetes Secrets are encrypted by default.
Correct
Secrets are base64-encoded, not encrypted. Anyone with access to etcd or the API server can decode them. Encryption at rest must be explicitly configured.
Mistake
NetworkPolicy denies all traffic by default.
Correct
If no NetworkPolicy selects a pod, all traffic is allowed. Only when a policy selects a pod does it become deny-by-default for that pod.
Mistake
PodSecurityPolicy is the current standard for pod security.
Correct
PodSecurityPolicy was deprecated in Kubernetes 1.21 and removed in 1.25. The replacement is Pod Security Admission (PSA) with Pod Security Standards (Privileged, Baseline, Restricted).
Mistake
Running containers as root inside the container is safe because of namespace isolation.
Correct
Root inside a container is not the same as root on the host, but if the container is privileged or has certain capabilities (e.g., SYS_ADMIN), it can perform privileged operations. Always run containers with a non-root user and drop all unnecessary capabilities.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
PodSecurityPolicy (PSP) was a deprecated Kubernetes resource that allowed fine-grained control over pod security contexts. It was removed in Kubernetes 1.25. Pod Security Standards (PSS) and Pod Security Admission (PSA) are the replacement. PSS defines three policy levels: Privileged (no restrictions), Baseline (minimal restrictions), and Restricted (strictest). PSA is an admission controller that enforces these policies at the namespace level using labels. The exam expects you to know that PSP is deprecated and PSA is the current standard.
Use the command `kubectl auth can-i create pods --as=system:serviceaccount:<namespace>:<sa-name>`. Replace <namespace> and <sa-name> with the actual values. This command checks RBAC permissions. If the output is 'yes', the service account can create pods. You can also check by describing the role binding: `kubectl describe rolebinding <binding-name> -n <namespace>`.
Even though root inside a container is not the same as root on the host (due to user namespace mapping), if the container runs with `privileged: true` or has certain capabilities (e.g., SYS_ADMIN, SYS_PTRACE), the process can perform actions that affect the host. Additionally, kernel vulnerabilities may allow container root to escape to the host. The best practice is to run containers with a non-root user (use USER directive in Dockerfile) and drop all capabilities except those required.
A default deny network policy denies all ingress and egress traffic to selected pods. To create one, define a NetworkPolicy that selects all pods in a namespace (podSelector: {}) and has no ingress or egress rules. For example: ```yaml apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: default-deny-ingress spec: podSelector: {} policyTypes: - Ingress ``` This policy will deny all ingress traffic to pods in that namespace. Similarly, you can create one for egress. Remember that if no policy selects a pod, all traffic is allowed.
Falco uses kernel modules or eBPF to capture system calls (syscalls) from all processes, including those in containers. It compares syscall events against a set of rules (e.g., 'Spawned a shell in a container', 'Read sensitive file'). When a rule matches, Falco generates an alert. For example, if a container runs `/bin/bash`, Falco can alert because that is unusual for a production container. Falco can output alerts to stdout, syslog, or integrate with SIEMs.
When a pod has `hostNetwork: true`, it uses the host's network stack directly instead of its own network namespace. This means the pod can access any network interface on the host and is not subject to Kubernetes network policies. It also bypasses any network isolation. This is a security risk and should only be used for system-level pods (e.g., kube-proxy, CNI plugins). Always question why a pod needs hostNetwork.
Use `kubectl get rolebindings --all-namespaces` and `kubectl get clusterrolebindings` to list all bindings. Then describe each binding to see the subjects and roles. For a deeper audit, use tools like `kube-bench` or `rbac-tool`. You can also use `kubectl auth can-i --list --as=<user>` to list permissions for a specific user. Regular audits help identify over-permissive roles.
You've just covered Container and Kubernetes Security Analysis — now see how well it sticks with free CS0-003 practice questions. Full explanations included, no account needed.
Done with this chapter?