Troubleshooting Scenario Questions

apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: allow-egress namespace: app spec: podSelector: {} egress: - to: - ipBlock: cidr: 192.168.1.100/32 policyTypes: - Egress

Why it fails: This egress rule targets all pods in the namespace and correctly uses ipBlock for the external database IP, but it omits the ports field. Without restricting to TCP/3306, the policy permits every destination port on that IP, which is broader than necessary and fails to demonstrate the precise least-privilege access required for the MySQL connection.

apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: allow-egress namespace: app spec: podSelector: matchLabels: app: myapp egress: - to: - ipBlock: cidr: 192.168.1.100/32 policyTypes: - Egress

Why it fails: By setting podSelector to matchLabels: app: myapp, this policy only applies to pods carrying that exact label. The developer's pod is not guaranteed to have that label, so if the selector matches no pods the egress rule is a no-op and the original connectivity failure remains. A namespace-wide selector (podSelector: {}) would cover all pods, including the affected one.

apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: allow-egress namespace: app spec: podSelector: {} ingress: - from: - ipBlock: cidr: 192.168.1.100/32 policyTypes: - Ingress

Why it fails: This NetworkPolicy defines an ingress rule, which filters traffic coming into the pods from the database IP, not traffic leaving the pods. Since the developer’s pod cannot initiate a connection to an external data source, the missing capability is egress. An ingress rule has no effect on outbound connectivity, so this policy does not address the reported problem.

The securityContext.seccompProfile.defaultRuntimeProfile field must be set to 'custom-profile.json'.

Why it fails: The seccompProfile API defines only `type` and `localhostProfile` as configurable fields; `defaultRuntimeProfile` does not exist. A custom JSON profile is attached by setting `type: Localhost` and specifying `localhostProfile: custom-profile.json`. Setting a nonexistent field would be rejected by API validation or ignored, not load the file.

The seccomp profile should be defined in the pod's annotations, not securityContext.

Why it fails: Kubernetes originally supported seccomp via pod annotations, such as `seccomp.security.alpha.kubernetes.io/pod`, but these were deprecated in favor of the native `securityContext.seccompProfile` field. In current releases, annotations are no longer honored for seccomp, so placing the profile reference there would have no effect. The correct declarative approach is to set `securityContext.seccompProfile.localhostProfile` and `type: Localhost`.

The localhostProfile field must be an absolute path.

Why it fails: The `localhostProfile` field must not be specified as an absolute path; it is interpreted relative to the node's seccomp profile root (usually `/var/lib/kubelet/seccomp`). For example, `custom-profile.json` resolves to `/var/lib/kubelet/seccomp/custom-profile.json`. Using an absolute path like `/var/lib/kubelet/seccomp/custom-profile.json` would cause the kubelet to concatenate it incorrectly, so the file would not be found.

Deploy Falco with a rule to detect outbound connections

Why it fails: Deploying Falco with a rule to detect outbound connections is a monitoring and detection mechanism, not an immediate forensic investigation step. Falco runs as a DaemonSet outside the pod and relies on kernel events, so it can only alert you to suspicious behavior after the fact, without providing the raw packet-level details needed to see exactly what data is being exfiltrated or which C2 endpoints are contacted. Setting it up takes time and does not capture the current traffic from the compromised pod, making it a poor first response for an active incident.

Create a NetworkPolicy to deny all egress traffic

Why it fails: Creating a NetworkPolicy to deny all egress traffic is a remediation or containment action, not an investigation technique. While it can stop the compromised pod from communicating with external hosts, it destroys the evidence of any ongoing connections and may alert the attacker that they've been discovered. It also provides no visibility into what traffic was already sent, and its effectiveness depends on the CNI plugin supporting NetworkPolicy, which is not guaranteed in every cluster.

Check the pod's logs using 'kubectl logs <pod>'

Why it fails: Checking the pod's logs with 'kubectl logs <pod>' is insufficient for investigating a suspected network compromise because application logs typically do not record low-level network connections unless the application explicitly writes them. Even if the application is legitimate, a compromised process can alter or suppress logs to hide its activities, leaving you without packet-level details like exact source to destination flows, packet payloads, or timing. Logs are useful for application-level anomalies, but they are not a substitute for capturing the actual network traffic.

The node has insufficient memory

Why it fails: Insufficient memory causes the kubelet to evict the pod or the container to be OOMKilled, producing events like 'The node was low on resource: memory' or an exit code 137 — never a message about the container runtime network. The error 'container runtime network not ready' is a sandbox-level failure indicating the CNI network setup for the pod's network namespace failed, which memory pressure does not affect. Memory exhaustion occurs after the pod is started and processes allocate memory, while the network not ready error occurs during pod sandbox creation, long before any container process runs.

The pod is trying to mount a hostPath volume

Why it fails: HostPath volumes are incompatible with the traditional local filesystem assumptions of runC, but Kata Containers explicitly supports them by sharing the host directory into the guest VM via virtiofs or 9p, with only minor restrictions such as the directory needing to exist. A problem with a hostPath volume would surface as a volume mount failure, e.g. 'failed to mount volume' or 'MountVolume.SetUp failed', not as a network readiness error from the container runtime. Since the reported error is specifically about network readiness, the selected option points to a CNI/network configuration failure, not storage.

The pod's securityContext sets runAsNonRoot: true

Why it fails: The runAsNonRoot security context only constrains the UID of the process running inside the container; it does not interact with the network stack or the CNI plugins. If the image's user is root and runAsNonRoot is true, the container runtime refuses to start the container with a message about 'runAsNonRoot' and the image's user, but this is a container-level security check, totally separate from the pod sandbox network setup. Therefore, this setting cannot produce the reported 'container runtime network not ready' error, which is caused by the sandbox's network interface failing to initialize during CNI configuration.

The webhook server uses HTTP instead of HTTPS

Why it fails: The API server uses the URL derived from clientConfig to dispatch admission requests. If the webhook server listens on plain HTTP but the clientConfig explicitly points to an http:// endpoint, the request will still be sent and the webhook will be invoked. A protocol mismatch—clientConfig expecting https while the server uses HTTP—would cause a TLS error, but the option merely states that HTTP is used by the server, not that transport negotiation failed. Therefore, HTTP by itself does not prevent the webhook from being called.

The webhook service is of type ClusterIP

Why it fails: A ClusterIP service is the typical and acceptable way to expose a webhook server to the API server within the same cluster. The API server runs in-cluster and can resolve ClusterIP services by their DNS names; no LoadBalancer or NodePort is required. Since admission webhooks are only called from the API server, internal ClusterIP reachability is sufficient, so this would not prevent the webhook from being invoked.

The webhook server returns a 502 status code

Why it fails: A 502 status code is a response returned by the webhook server after the API server has already reached it and completed an HTTP round trip. This proves the webhook was called; the admission controller simply received an error response, which may cause a denial or fallback according to failurePolicy, but it does not explain a failure to call the webhook at all. Therefore, this cannot be the reason the validatingwebhookconfiguration is not working as expected.