Monitoring, Logging and Runtime Security practice questions

crictl logs <container-id>

Why it fails: crictl logs retrieves only the standard output and standard error streams produced by the container's main process, which reveals application-level messages but provides no access to the container's filesystem. Logs are stored by the runtime or a logging driver and do not reflect deleted files, configuration artifacts, or binaries present on disk. For filesystem forensic analysis, capturing the root filesystem via crictl export is required.

crictl inspect <container-id>

Why it fails: crictl inspect returns a JSON object containing the container's configuration, status, mounts, network settings, and resource details, but it does not expose the actual content of files on the container's filesystem. It can be useful to identify the image and mount points, yet it cannot answer questions about file contents or hidden artifacts. To inspect raw files, you must export the filesystem or use a copy-on-write tool.

crictl exec <container-id> tar cvf /tmp/fs.tar /

Why it fails: crictl exec runs a command inside the container, which creates a new process and thus changes the runtime state; using tar inside the container writes an archive to /tmp/fs.tar, leaving forensic artifacts and potentially triggering anti-forensic mechanisms in a compromised pod. Additionally, the archive remains inside the container, requiring a second exec to copy it out, and the tar binary itself may be subverted by an adversary. A proper acquisition should be performed externally using crictl export to guarantee the container's filesystem is not modified.

Modify the rule condition to add `and not user.name in (admin, root)`

Why it fails: This would exclude all admin and root users, which might be too broad and miss real threats from those accounts.

Change the rule priority to `INFO`

Why it fails: Changing priority does not reduce the number of alerts; it only changes the severity level.

Disable the rule

Why it fails: Disabling the rule would stop detecting the activity entirely, which may not be desired.

The emptyDir volume is not mounted correctly in the pod spec

Why it fails: If the volume were not mounted, the container would fail to start or the directory would be empty, but the issue is permissions.

The emptyDir volume is using a read-only storage class

Why it fails: emptyDir volumes are ephemeral and always writable; there is no storage class.

readOnlyRootFilesystem also applies to mounted volumes

Why it fails: readOnlyRootFilesystem only affects the root filesystem of the container, not mounted volumes.

proc.name contains 'shadow'

Why it fails: proc.name matches the process binary name, so a process called shadow would trigger while an actual open of /etc/shadow by cat or bash would not. It is tempting because the filename appears in the field, but detection requires the file descriptor path fd.name, not the executable's name.

container.id != host and fd.name=/etc/shadow

Why it fails: The container.id != host clause excludes host activity but the condition omits an event type, so it matches any syscall touching that path rather than the open attempt specified. It is tempting because it scopes to containers, yet the stem asks for the open event, which requires evt.type=open paired with fd.name.

evt.type=read and fd.name=/etc/shadow

Why it fails: evt.type=read fires when data is read from an already-open descriptor, missing the open syscall itself. It is tempting because reading /etc/shadow is the sensitive act, but the stem asks to detect attempts to open the file, which requires evt.type=open with fd.name=/etc/shadow.

--audit-webhook-config=/etc/kubernetes/audit-webhook.yaml

Why it fails: This configures a webhook audit backend, not the policy file.

--audit-log-path=/var/log/audit.log

Why it fails: This sets the path for audit log output, not the policy.

--audit-log-format=json

Why it fails: This sets the log format, not the policy or level.

Use 'crictl export <container-id>' to create a tar archive of the container's filesystem

Why it fails: 'crictl export' produces a tar stream of the container's union filesystem, which does include the writable layer and any persisted files. However, it deliberately excludes RAM (memory pages), process descriptors, kernel state, and ephemeral network connections, all of which may hold crucial evidence such as open sockets or process memory. The collection is also performed while the container is live, so the act of exporting can alter the filesystem, and the missing memory component makes it a partial, not a full, forensic capture.

Use 'kubectl cp' to copy files from the container to the node

Why it fails: 'kubectl cp' uses the Kubernetes API's exec mechanism to run tar inside the target container and stream a limited set of user-specified paths out to the client or node. It does not and cannot access memory or kernel state, and because it executes a process inside the container, it perturbs the evidence and may trigger anti-forensic timers. Moreover, it relies on a running container and specific paths you already know about, so it is the opposite of a comprehensive, non-invasive forensic collection method.

Use 'crictl save' to save the container image

Why it fails: The 'crictl save' command (analogous to 'docker save') serializes the read-only image layers into an archive, yielding only the pristine base image. It does not capture the container's writable layer—where runtime modifications, logs, and deleted-file remnants reside—nor any in-memory artifacts such as decrypted secrets or running process state. Consequently, even if the command succeeded, the resulting artifact would miss most of the dynamic forensic evidence and cannot serve as a complete container snapshot.

evt.type=execve and proc.name in (bash, sh) and container.id != host

Why it fails: This condition catches every bash or sh execution occurring anywhere in a container, because container.id != host merely states that the event happened inside a container, regardless of which process is involved. The most critical flaw is the missing proc.pid=1 predicate: a normal shell launched later in a running container (e.g., via kubectl exec or a user shell inside the container) would also trigger this rule, creating excessive false positives. Additionally, using evt.type=execve directly is less robust than the spawned_process macro, which also covers execveat and verifies the event direction; however, the primary reason this is wrong is the absence of the PID 1 requirement.

container.id != host and proc.name in (bash, sh) and proc.ppid=0

Why it fails: The proc.ppid=0 condition is a fragile and indirect stand-in for identifying PID 1. In a Linux PID namespace, the parent of the container init process is typically 0, but not all shells that run as PID 1 have ppid=0 under every runtime setup, and a non-PID-1 process that gets reparented to init may also momentarily have ppid=0 or a different value, leading to both false negatives and false positives. This rule also lacks the spawned_process macro to ensure the event is the actual process-creating syscall, so it could match a process that already existed before Falco started monitoring. The direct and dependable check is proc.pid=1, not an inferred parent PID condition.

proc.name = bash and proc.pid=1

Why it fails: This rule correctly targets the shell name and PID 1, but it omits any event-type or syscall filter, meaning Falco would evaluate it against every event related to a process named 'bash'. Since Falco rules are event-driven, without spawned_process (or evt.type=execve and evt.dir=<) the rule could match an already-running process that existed before Falco started monitoring, or a process that was created by a fork that never performed an exec. It is also overly restrictive because it only matches 'bash', completely missing 'sh', which is equally common as a container init shell; the correct rule must include both shells and the execve-based event filter.

kubectl get svc <service-name> -o yaml

Why it fails: kubectl get svc <service-name> -o yaml outputs the Service resource definition, including its label selector and port mappings, but it never contains the resolved endpoint IPs. The Service object is only the desired-state declaration; the actual pod IPs are reconciled into a separate Endpoints object by the service controller. Therefore, even if the YAML looks correct, you cannot see whether any pods are currently selected, ready, or reachable, so it is not the fastest or most direct way to confirm external traffic routing.

kubectl get pods -l <selector>

Why it fails: kubectl get pods -l <selector> lists pods that match that exact label query, but you must first know the Service's selector and manually compare. Even when pods appear, they might be in a Pending or CrashLoopBackOff state, or their ports might not match the Service's targetPort, so the Service would still have no usable endpoints. This command gives you the pod side of the equation only, not the Service's actual endpoint resolution, making it insufficient to prove that the NodePort Service is reachable from outside.

kubectl describe service <service-name>

Why it fails: While kubectl describe service <service-name> does display the current Endpoints field, it also mixes in selector, port, event, and annotation data, making it more verbose than needed for a quick connectivity check. For a NodePort Service that is unreachable, the fastest and most unambiguous command is kubectl get endpoints, which returns just the endpoint IPs and ports. The describe output could still be used, but it is not the most direct tool for the specific symptom of missing or incorrect endpoints.

Modify the container image to disable networking

Why it fails: Rebuilding the container image to strip networking is a deployment-time remediation, not a runtime containment action. The currently running pod retains its existing image and network stack until it is replaced, so it provides no immediate isolation. Moreover, altering the image may destroy the exact runtime artifacts needed for forensic analysis and cannot prevent an already active attacker from continuing within the live pod.

Delete the pod immediately to stop the threat

Why it fails: Deleting the pod outright is a destructive response that terminates the container, removes its writable layer, and erases ephemeral logs and memory dump possibilities, which can compromise forensic evidence collection. It also fails to address the underlying compromise because the attacker's command-and-control may simply cause a redeployment, and the same vulnerability or access persists. In incident response, isolation is meant to contain and preserve a subject for analysis, not to destroy it; immediate deletion may actually increase the attacker's advantage by removing audit trails.

Use two rules with 'level: Request' and 'level: Metadata' without default

Why it fails: This policy fails to provide a default audit level. In Kubernetes audit policy, if no rule matches a request and no default level is specified, that request is not logged at all. Merely listing two rules with 'level: Request' and 'level: Metadata' without any selectors (like namespaces) makes them ambiguous and effectively redundant—the first rule matches everything, so the second is never reached. Alternatively, if selectors were added to scope them, any request not matching those selectors would be silently skipped, leaving gaps in the audit log. A correct policy must use a default level to catch all unmatched requests, as the correct answer does.

Default level Request with a rule for kube-system at Metadata

Why it fails: Setting the default level to Request would log every request at Request level, including the full request body for all namespaces. This is more verbose than required and exposes sensitive data unnecessarily. Then adding a rule for kube-system at Metadata would downgrade kube-system requests to Metadata, which is the opposite of the requirement—kube-system needs Request-level logging, not Metadata. In audit policy, rules override the default, so this configuration yields Request for everything except kube-system (Metadata) rather than Metadata for everything except kube-system (Request). Therefore, this option inverts the intended logging levels.

%pod.name

Why it fails: %pod.name is not a valid Falco output field. The correct field for Kubernetes pod name is %k8s.pod.name.

%fd.name

Why it fails: %fd.name is not a valid Falco output field; it is used in rule conditions for file descriptors but not in the output format string.

%k8s.ns

Why it fails: %k8s.ns is not a valid Falco output field; the correct field is %k8s.ns.name for namespace.

RequestProcessed

Why it fails: RequestProcessed is not a valid Kubernetes audit stage. The official audit policy supports exactly four stages: RequestReceived, ResponseStarted, ResponseComplete, and Panic. RequestProcessed likely stems from confusion with the RequestReceived stage, which fires when the kube-apiserver first receives the request, before any processing begins. Since it does not exist, any audit policy referencing it will fail validation, so it cannot be a correct answer.

open

Why it fails: The open syscall merely opens a file descriptor for reading or writing; it does not alter the process's user or group credentials. While opening sensitive files like /etc/shadow can facilitate later attacks, the syscall itself cannot change UID/GID and therefore is not the point where privilege escalation occurs. Monitoring open typically targets file disclosure, not credential changes.

execve

Why it fails: execve replaces the current process image with a new program, and although executing a setuid-root binary results in an effective UID change, that change is performed by the kernel's exec-time setuid-bit logic, not by the execve syscall itself. The actual syscall that explicitly manipulates process credentials at runtime is setuid. Falco rules for privilege escalation should therefore key on setuid/setgid invocations rather than generic execve calls, unless the rule specifically tracks execution of known setuid binaries.

connect

Why it fails: connect establishes an outbound or inbound network connection on a socket; it has no effect on the process's real, effective, or saved user IDs. A malicious connection may be part of a reverse shell that follows privilege escalation, but the connect call neither grants nor raises privileges. Detecting connect is useful for network egress monitoring, not for identifying the credential-changing event.

Apply an egress rule on the target pod with 'to' podSelector matching 'app: frontend'

Why it fails: Egress rules govern traffic leaving a pod, so applying an egress rule on the target pod with 'to' podSelector matching 'app: frontend' would only restrict which destinations the target pod can connect to. It does nothing to control which sources are allowed to send traffic inbound to the target pod. To restrict inbound traffic, you must use an ingress rule on the pod that receives the traffic, not an egress rule on that same pod.

Apply an egress rule on the source pods with 'to' podSelector matching the target pod

Why it fails: Applying an egress rule on the source pods with 'to' podSelector matching the target pod restricts which destinations those source pods can reach, but it does not enforce that the target pod only receives traffic from 'app: frontend'. This rule only affects the source pods you select; any other pod not covered by the rule could still send traffic to the target if no default-deny ingress policy exists. The requirement is about controlling inbound access to the target pod, which is inherently an ingress concern, not an egress restriction on potential clients.

Apply an ingress rule on the source pods with 'from' podSelector matching the target pod

Why it fails: An ingress rule on the source pods with 'from' podSelector matching the target pod controls what traffic is allowed into the source pods, not into the target pod. This would allow the source pods to receive traffic from the target, but it says nothing about who can send traffic to the target pod. The target pod's ingress is what needs to be filtered to ensure only 'app: frontend' can send it traffic, and that requires an ingress policy on the target pod itself.

--enable-admission-plugins

Why it fails: The --enable-admission-plugins flag activates admission controllers such as PodSecurity or NodeRestriction, which intercept API requests after authentication/authorization and can mutate or reject them before they are persisted. This is a request-modification mechanism, not a recording mechanism, so it has no effect on audit logging. Audit logging is about generating an audit trail of requests, which requires the audit policy to be defined separately; therefore this flag does not and cannot enable audit logging.

--authorization-mode

Why it fails: The --authorization-mode flag configures how the API server decides whether a principal is permitted to perform a request, commonly with values like RBAC, AlwaysAllow, or ABAC. This is a gatekeeping decision point and, while audit events may capture the authorization result, the flag itself does not generate or enable any audit log output. Enabling audit logging is a separate concern that requires a defined audit policy to select which events to record and at what detail level, so this flag is not the one that enables audit logging.

--audit-log-path

Why it fails: The --audit-log-path flag sets the file system path where audit log entries would be written, but by itself it does not actually cause any audit events to be generated. In Kubernetes, the audit backend only becomes active when --audit-policy-file is also supplied; without a policy file, the default audit policy is 'None', meaning no events are recorded, so the log file remains empty or is never written. Thus this flag is necessary for log output but insufficient to enable audit logging, because the policy defines what is logged and at which audit level.

Authentication

Why it fails: Authentication is not an audit stage; it occurs before auditing.

Authorization

Why it fails: Authorization is not an audit stage; it occurs after authentication.

INFO

Why it fails: Falco's INFO priority is the second-lowest severity and is reserved for benign, expected events such as successful user logins, namespace changes, or periodic health checks. In a typical Kubernetes deployment, these events are useful for audit trails but are not considered suspicious on their own, and alerting on them would create excessive false-positive noise. Security responders expect malicious indicators to appear at WARNING or above, so INFO cannot signify a potentially malicious activity.

ERROR

Why it fails: ERROR priority in Falco indicates that a system call, rule, or internal component failed during execution — for instance, a permission error, a broken pipe, or an invalid argument — but this failure does not imply attacker intent. Malicious behavior often succeeds silently or uses errors as part of an exploit chain, but the ERROR priority itself is defined for operational faults, not security judgement. Since the question asks about a potentially problematic event, ERROR is too ambiguous and lacks the 'investigate as suspicious' connotation that WARNING carries.

DEBUG

Why it fails: DEBUG is the most verbose and lowest meaningful priority, designed for tracing rule evaluation, variable expansion, and system call details while developing or troubleshooting Falco rules. It is never used to classify a security event for alerting because its output is overwhelmingly technical and meant for rule authors, not defenders responding to incidents. Consequently, a DEBUG event contains no assertion of maliciousness and would be filtered out in production before any investigation.

kubectl label pod pod-name isolate=true

Why it fails: The label operation only modifies pod metadata; it has no inherent enforcement mechanism. Kubernetes network isolation is implemented solely via NetworkPolicy objects, which use label selectors to identify the pod and then explicitly deny or allow traffic. Without applying a NetworkPolicy that selects pod-name, the isolate=true label is inert and the pod remains fully accessible, so this command does nothing to contain the incident.

kubectl cordon node-name && kubectl drain node-name

Why it fails: Cordoning the node prevents new pod scheduling, but draining evicts existing pods, including the compromised pod. The eviction terminates the container, destroying volatile evidence such as running processes, memory contents, and open network sockets. Moreover, if the pod is controlled by a ReplicaSet or Deployment, it will be recreated elsewhere, potentially spreading the compromise and confounding the investigation, all of which is the opposite of forensic preservation.

kubectl delete pod pod-name

Why it fails: Deleting the pod issues a termination signal to the container runtime, which stops the container and removes its writable layer, including logs, process artifacts, and any malicious files. Without a preceding forensic image or collection, this destroys the very evidence needed to determine the attack vector and attacker actions. The pod may also be immediately replaced by its controller, overwriting the original identity and further complicating the investigation. Incident response demands preserving the pod or its artifacts, not deleting them.

ResourceQuota

Why it fails: ResourceQuota is a namespaced admission-control object that aggregates and caps the total compute and storage resources (CPU, memory, persistent storage claims, and object counts) that a namespace can consume. It operates at the cluster management layer and has no mechanism to alter the mount options or permissions of an individual container's filesystem. A container's root filesystem can be made read-only only through a SecurityContext setting such as readOnlyRootFilesystem: true, which is a per-container runtime attribute unrelated to quota-based resource limits.

PodSecurityPolicy

Why it fails: PodSecurityPolicy was a cluster-level admission policy that could enforce readOnlyRootFilesystem as one of many security constraints, but it applied only at pod creation time via a matching policy and required a PodSecurityPolicy admission controller. It is deprecated since Kubernetes 1.21 and removed in 1.25, making it unusable in current clusters. More importantly, setting a read-only root filesystem today is done declaratively on the Pod/Container via SecurityContext, not through a separate admission policy resource, so PodSecurityPolicy is the wrong direct mechanism.

NetworkPolicy

Why it fails: NetworkPolicy is a namespaced resource that defines ingress and egress firewall rules between pods and external endpoints, using pod labels and port selectors to allow or deny network traffic. It operates at layer 3/4 of the network stack and has no influence on the container runtime's mount table or filesystem permissions. Making a container's root filesystem read-only is enforced by the container runtime when starting the container, based on the SecurityContext, not by any network policy.