Workloads & Scheduling practice questions

Pod priority and preemption

Why wrong: Pod priority and preemption are mechanisms that influence the scheduling order of pods and allow higher-priority pods to evict lower-priority pods from nodes if resources are scarce. While critical for workload management and ensuring important applications run, these features do not provide a direct mechanism for a pod to express a preference or requirement for specific *types* of nodes based on labels or attributes. They manage *when* a pod gets scheduled, not *where* it specifically targets.

Taints and tolerations

Why wrong: Taints are applied to nodes to repel pods, preventing them from scheduling onto that node unless the pod has a corresponding toleration. This mechanism is primarily used for ensuring nodes are reserved for specific workloads or to keep misbehaving pods off certain nodes. However, taints and tolerations do not actively *attract* pods to particular nodes; they only allow pods to *tolerate* an existing exclusion, making them unsuitable for expressing a preference to land on a specific node type.

Resource quotas

Why wrong: Resource quotas are a mechanism to limit the aggregate resource consumption (like CPU, memory, or number of pods) within a specific Kubernetes namespace. They ensure fair usage among teams or projects and prevent any single namespace from consuming all available cluster resources. However, resource quotas do not provide any functionality to influence *which specific nodes* a pod will be scheduled onto; they only enforce limits on *how much* can be consumed within a defined scope, irrespective of node characteristics.

The deployment has a resource quota that prevents new pods

Why wrong: If a ResourceQuota limit were reached in the namespace, the API server would reject the creation of any new Pods entirely. However, this does not explain why a specific third replica remains in an unevicted state on a failed node while the deployment controller is still expecting three replicas. Furthermore, a quota violation would trigger a ReplicaSet event indicating a failure to create the pod, rather than leaving the existing pod bound to the dead node.

The pod's terminationGracePeriodSeconds is set to 0

Why wrong: Setting terminationGracePeriodSeconds to 0 forces immediate deletion of the pod from the API server once the deletion signal is received. This configuration accelerates the rescheduling process rather than delaying it, as it bypasses the standard graceful shutdown window. It cannot cause a pod to linger on an unresponsive node before eviction is initiated by the control plane.

The deployment's replicas field is set to 2

Why wrong: The scenario explicitly states that the deployment is configured with 3 replicas. If the replicas field in the deployment specification were actually set to 2, the controller would only maintain two running pods under normal conditions. This mismatch does not explain why a third pod exists but is failing to reschedule after a node failure.

Random pods

Why wrong: StatefulSet scale-down is deterministic, never random. When the desired replica count is reduced, the controller always removes pods in reverse ordinal order, starting from the highest ordinal (n-1) and proceeding downward. Random selection would violate the stable per-pod identity and matching PersistentVolumeClaims that StatefulSet guarantees, because those identities are bound to specific ordinals.

Pods with the lowest ordinals (0 and 1)

Why wrong: The lowest ordinals are the ones that survive a scale-down, not the ones removed. StatefulSet deliberately retains the smallest ordinals to maintain a contiguous range from 0 to (replicas-1) after the operation. Deleting pods 0 and 1 would create gaps (e.g., leaving ordinals 2, 3, 4) and break the stable identity model, so the controller would never choose them as the first to remove.

Pods with the highest resource usage

Why wrong: Resource usage is irrelevant to StatefulSet scale-down. The controller does not inspect CPU or memory utilization; it only checks the desired replica count against the current number of pods and deletes the highest ordinals to meet that count. Pod eviction based on resource pressure is handled by the kubelet or cluster autoscaler, not the StatefulSet controller's scaling logic.

Taint on non-GPU nodes and toleration on the pod

Why wrong: Taints and tolerations are primarily designed to repel pods from nodes unless the pod explicitly tolerates the taint. Applying a taint to non-GPU nodes would prevent pods without the corresponding toleration from scheduling there. However, this mechanism does not attract pods to GPU nodes or guarantee that a pod must run on a GPU node; it only allows the pod to run on any node it tolerates, including non-GPU nodes if they are untainted. The goal is to require a GPU node, which taints alone cannot achieve.

Pod affinity to prefer GPU nodes

Why wrong: Pod affinity rules are designed to schedule pods on nodes that already have other specified pods running on them, based on labels of those pods. While preferredDuringSchedulingIgnoredDuringExecution could express a preference, it's based on the presence of other pods, not direct node labels or inherent hardware capabilities like a GPU. Therefore, using pod affinity to select nodes based on their intrinsic hardware features is an incorrect application of this feature, as it doesn't directly evaluate node labels.

Node affinity with a requiredDuringSchedulingIgnoredDuringExecution rule for gpu=true

Why wrong: Node affinity with `requiredDuringSchedulingIgnoredDuringExecution` fails because the `IgnoredDuringExecution` component means that if a node's `gpu=true` label is removed *after* the pod is scheduled, the pod will continue running on that node, violating the application's continuous GPU requirement. This rule is tempting as `requiredDuringScheduling` ensures initial placement. It is suitable when initial placement is critical, but subsequent label changes during execution are acceptable or not anticipated, preventing unnecessary pod evictions.

Delete the master node from the cluster

Why wrong: Removing the master node entirely from the cluster is an extreme and destructive action that would dismantle the control plane, rendering the cluster unmanageable. To prevent pod scheduling, administrators must use scheduling constraints like taints rather than deleting critical infrastructure components that run the API server, scheduler, and controller manager.

Use a resource quota on the master namespace

Why wrong: ResourceQuotas are admission control policies designed to limit total resource consumption (like CPU, memory, or object counts) within a specific Kubernetes namespace. They operate at the namespace level and cannot influence the scheduler's node-selection decisions or prevent pods from being scheduled onto specific physical or virtual nodes.

Set nodeSelector on the master node

Why wrong: The nodeSelector field is a pod specification property used to constrain pods to nodes with particular labels, not a configuration option applied directly to nodes. Attempting to configure a node selector on a node object is syntactically invalid and does not prevent arbitrary pods from being scheduled onto that node.

The pod has resource requests that exceed available capacity

Why wrong: If a pod's CPU or memory requests exceeded the allocatable capacity of the nodes, the scheduler would emit events explicitly stating "Insufficient cpu" or "Insufficient memory". Because the kubectl describe pod output specifically highlights taint-related scheduling failures, resource constraints are not the root cause of this pending state.

The nodes are cordoned

Why wrong: Cordoning a node marks it as unschedulable by applying the Spec.Unschedulable field, which triggers a specific "node(s) were unschedulable" or "node(s) cordoned" event from the scheduler. This is distinct from taint-matching failures, which generate events pointing directly to unmatched taints rather than general unschedulable node status.

The kube-scheduler is not running

Why wrong: If the kube-scheduler control plane component were completely offline, no scheduling cycle would occur for the pending pod. Consequently, the kubectl describe pod output would show a complete absence of scheduling events or decisions, rather than active warning events detailing why scheduling failed.

Define a nodeSelector in the pod spec

Why wrong: A `nodeSelector` matches nodes by static label key-value pairs, but the scenario requires scheduling only onto a node that physically possesses a specific custom resource such as a license dongle. This approach fails because a node label does not dynamically reflect hardware presence; the dongle could be removed without the label changing. It is tempting because `nodeSelector` is the simplest mechanism for directing pods to nodes with particular attributes, and it would be correct if the requirement were merely a static node property like an SSD or GPU, not a removable physical device.

Assign the pod a high priority so it gets scheduled on any node

Why wrong: Assigning a pod a high priority only influences its position in the scheduling queue and its ability to preempt lower-priority pods if resources are scarce. It does not provide any mechanism to direct the pod to a specific node with particular hardware or software resources. A high-priority pod will still be scheduled on *any* available node that meets its resource requests, not necessarily one with the specialized resource needed for the batch job.

Use pod anti-affinity to avoid nodes without the resource

Why wrong: Pod anti-affinity is designed to prevent pods from being scheduled on the same node as other specified pods, primarily for high availability or workload isolation. It focuses on the relationship *between pods* rather than the inherent characteristics of the *node itself*. While it can indirectly influence node choice by excluding nodes already running certain pods, it cannot directly select nodes based on the presence or absence of a specific hardware resource like a license dongle.

Use a toleration for a taint that exists only on nodes in the zone

Why wrong: Tolerations are not a selection mechanism; they merely allow a pod to be scheduled onto nodes that carry a matching taint. If a zone's nodes are tainted, a toleration lets the pod pass the taint check, but the scheduler is free to place it on any other tainted node anywhere in the cluster. Tolerations only relax restrictions—they never force or even prefer scheduling onto the nodes with the taint, so they cannot guarantee zone selection.

Use pod affinity to attract pods to nodes in that zone

Why wrong: Pod affinity is designed to co-locate pods relative to other pods, not to select nodes based on node labels. It uses label selectors against pods already running in the cluster, meaning it can only influence placement if there is an existing pod with matching labels in that zone, and it is often soft (preferred) rather than required. This indirect, pod-relative mechanism is not a valid way to ensure a pod lands on a node in a specific zone based on node properties.

Set spec.nodeName to the name of a node in the zone

Why wrong: Setting spec.nodeName directly assigns the pod to a specific node by name, completely bypassing the scheduler and all admission checks. This approach requires knowing the exact node hostname, not the zone label, and it cannot react to node health or resource availability—the pod will be stuck if that node fails or is full. It is a hard override for special use cases like system pods, not a zone-based scheduling method.

DaemonSets can be scaled down by reducing the number of replicas

Why wrong: DaemonSets do not have a `replicas` field; therefore, scaling by reducing a replica count is impossible. The number of pods is determined by the set of nodes that match the pod template's nodeSelector and tolerations, not by a desired count. To reduce the number of pods, you must modify the nodeSelector to exclude certain nodes or add taints that prevent scheduling, not adjust a scalar replica field.

A DaemonSet can be used to run a batch job that completes

Why wrong: DaemonSet pods are intended for long-running services that should persist on every node, and they are not suited for batch jobs that finish and exit. If a DaemonSet pod completes successfully, the controller will restart it to maintain the desired state of one pod per node. For workloads that run to completion, the appropriate resource is a Job (or CronJob for scheduling), which manages pod lifecycle until all tasks finish.

The application is a batch job that should run to completion

Why wrong: StatefulSet is designed for long-running services that maintain state and continue serving requests; its pods are expected to remain alive. A batch job that runs to completion is exactly the opposite, and Kubernetes provides the Job controller specifically to track successful termination and manage retries. If you used a StatefulSet for a batch workload, the completion would be treated as a failure and the pod would be restarted indefinitely, breaking the desired run-to-completion semantics.

The application needs to run on all nodes in the cluster

Why wrong: Running one pod on every node in the cluster is the purpose of a DaemonSet, not a StatefulSet. DaemonSets are typically used for node-level infrastructure such as kube-proxy, log collectors, and monitoring agents. StatefulSet schedules a specified number of replicas according to desired count and placement constraints; it does not guarantee host coverage, so this requirement points away from StatefulSet.

The application is stateless and can be scaled arbitrarily

Why wrong: A stateless application that can scale arbitrarily is the exact use case for a Deployment, not a StatefulSet. StatefulSets guarantee stable, unique network identities and persistent storage per pod, which is unnecessary and wasteful for stateless workloads. This option is tempting because Deployments and StatefulSets both manage pods, but the choice hinges on whether the application requires ordered startup, sticky identity, or PVC binding — none of which apply here.

Frontend: Deployment, Service (NodePort); Backend: Deployment, Service (ClusterIP); no NetworkPolicy

Why wrong: While using a NodePort Service exposes the frontend on a static port across each node, this configuration completely lacks a NetworkPolicy to secure the backend. Without a NetworkPolicy, the backend ClusterIP Service remains open to unauthorized traffic from any pod within the Kubernetes cluster, failing to meet basic isolation requirements.

Frontend: Deployment, Service (ClusterIP); Backend: Deployment, Service (ClusterIP); NetworkPolicy to allow ingress from frontend namespace

Why wrong: Exposing the frontend via a ClusterIP Service restricts its accessibility solely to clients inside the cluster. Because the frontend application requires external access to serve public users, this configuration fails because ClusterIP does not provision an external IP address or route external traffic.

Frontend: Deployment, Service (LoadBalancer); Backend: Deployment, Service (LoadBalancer); NetworkPolicy to allow only frontend to backend

Why wrong: Utilizing a LoadBalancer Service for the backend is a severe security misconfiguration because it provisions a public-facing cloud load balancer, exposing the database or API directly to the internet. The backend should only be reachable internally via a ClusterIP, making this architecture unnecessarily vulnerable and costly.

The Pod was evicted due to resource pressure.

Why wrong: If a Pod is evicted due to resource pressure, such as disk or memory exhaustion, its status changes to Failed or Terminating as the kubelet actively terminates its running containers. A newly deployed Pod stuck in the Pending state has never been scheduled or started on a node, meaning it was not evicted.

The Pod requests more CPU than available on the node.

Why wrong: While excessive CPU requests can cause a Pod to remain Pending with an 'Insufficient cpu' event, the scenario specifically indicates the node is flagged with a disk pressure taint. The scheduler filters out nodes based on taints first, meaning the taint mismatch prevents scheduling before CPU capacity is even evaluated as the primary blocker.

The scheduler failed to communicate with the API server.

Why wrong: A complete communication failure between the kube-scheduler and the kube-apiserver would halt scheduling cluster-wide, leaving all new Pods in a Pending state. Since this is an isolated issue with a specific Pod deployment on a node with disk pressure, the control plane's scheduling loop is functioning normally.

Use podAffinity to prefer nodes running other Pods.

Why wrong: Pod affinity is designed to schedule Pods relative to other Pods based on labels, rather than targeting specific node hardware characteristics like SSDs. Additionally, using a 'preferred' affinity rule (soft affinity) does not guarantee placement, meaning the scheduler could still place the Pod on a non-SSD node if resources are constrained.

Set spec.nodeName to an SSD node's name.

Why wrong: Hardcoding a specific node name in spec.nodeName bypasses the Kubernetes scheduler entirely and binds the Pod to that single node. If that specific node fails, undergoes maintenance, or runs out of resources, the Pod will not be rescheduled onto another SSD-capable node, creating a single point of failure.

Add a taint to nodes without SSD and add a toleration to the Pod.

Why wrong: Taints and tolerations are used to repel Pods from nodes, not to attract them. While adding a taint to non-SSD nodes prevents this Pod from landing on them, it does not force the scheduler to choose an SSD node over other untainted nodes that might also lack SSDs but aren't tainted.

Add a toleration for 'arch=amd64:NoSchedule' to the Pod.

Why wrong: Adding a toleration for 'arch=amd64:NoSchedule' to a Pod would only be relevant if amd64 nodes were specifically tainted with `arch=amd64:NoSchedule`. Tolerations allow pods to be scheduled on nodes that possess a matching taint, but they do not actively *force* scheduling onto nodes of a particular architecture. To restrict pods to amd64 nodes, a taint would typically be applied to *non-amd64* nodes, and the pod would *not* tolerate that specific taint, or more directly, `nodeSelector` should be used.

Add a label 'kubernetes.io/arch: amd64' to all amd64 nodes and use nodeSelector.

Why wrong: The `nodeSelector` field in a Pod's specification is the most straightforward and effective method to constrain a Pod to run exclusively on nodes with specific labels. Kubernetes automatically applies the `kubernetes.io/arch: amd64` label to nodes possessing an amd64 architecture. Therefore, utilizing `nodeSelector` with this standard and readily available label precisely targets and ensures the Pods are scheduled solely on amd64 nodes, directly fulfilling the team's requirement.

Use podAntiAffinity to avoid scheduling on arm64 nodes.

Why wrong: Pod anti-affinity rules are specifically designed to prevent pods from being scheduled on the same node as other specified pods, or on nodes that already host certain pods. This mechanism evaluates the labels of *other pods* on a node, not the node's inherent attributes like CPU architecture. Consequently, `podAntiAffinity` cannot be used to directly avoid scheduling on nodes based on their architectural type, such as arm64 nodes, as it operates on a different scheduling dimension.

Set a runtimeClassName for amd64 in the Pod spec.

Why wrong: Setting a `runtimeClassName` in the Pod spec dictates which container runtime (e.g., `runc`, `gVisor`, `Kata Containers`) the Kubelet should use for that particular pod. This choice influences the container's isolation and performance characteristics, but it is entirely independent of the node's underlying CPU architecture (like amd64 or arm64). Therefore, specifying a `runtimeClassName` for 'amd64' would not constrain pod scheduling to amd64 nodes, as it serves a distinct purpose within the Kubernetes ecosystem.

DaemonSet

Why wrong: A DaemonSet ensures that all or some cluster nodes run a copy of a Pod, which is typically used for background system services like log collectors or monitoring agents. It does not manage finite batch tasks or run workloads to completion. Using a DaemonSet for a queue-processing batch job would inappropriately scale the workload based on cluster node count rather than queue size.

CronJob

Why wrong: A CronJob is used to run Jobs on a repeating, time-based schedule, similar to cron tasks in Linux. While it manages Jobs, it is designed for periodic execution rather than executing a one-off batch process triggered by queue availability. For a standard, non-scheduled queue processing task, a direct Job resource is the correct abstraction.

Deployment

Why wrong: Deployments are designed for stateless, long-running services like web applications that need to maintain a continuous state of availability. If a Pod in a Deployment exits successfully after processing a queue, the Deployment controller will interpret this as a drift and automatically restart it. This behavior creates an infinite loop of execution, which is unsuitable for finite batch processing.

Between 2 and 3

Why wrong: The range 2–3 ignores the maxSurge field, which permits the Deployment to create extra pods above the desired 3 before terminating old ones. With maxSurge=1, for example, the controller can temporarily run 4 pods total, so the upper bound is 4, not 3. Thus "between 2 and 3" is too restrictive and only part of the valid span.

Exactly 4

Why wrong: Exactly 4 is only the peak if maxSurge equals 1 and is not a constant state. Kubernetes does not force the number of pods to stay at 4; it only enforces the constraint that the total never exceeds replicas plus maxSurge. Depending on maxUnavailable, the count can also be 3 (no surge active) or even 2 (when one pod is taken down), making "exactly 4" an inaccurate fixed target.

Always 3

Why wrong: The Deployment never promises a static pod count during updates; rolling updates are designed to transition between ReplicaSets by creating new pods and removing old ones. If maxSurge=0 and maxUnavailable=1, the count can drop to 2; if maxSurge=1, it can rise to 4. Therefore, "always 3" is only true outside the update window and contradicts the rolling update mechanism itself.

Adding a toleration for a taint on the node.

Why wrong: A toleration only marks a pod as willing to be scheduled onto nodes that have a matching taint; it does not select or restrict the pod to any particular node based on labels. Without a matching taint, tolerations are ignored entirely, and even with a match, the scheduler still evaluates all nodes for placement. Therefore, it cannot be used as a mechanism to constrain a pod to nodes with specific labels.

Setting spec.nodeName to a node's name.

Why wrong: Setting spec.nodeName directly assigns the pod to a specific node by its name, completely bypassing the scheduler and label-based selection. This is a forced placement, not a restriction based on node labels, and it can cause scheduling failures if that node is unavailable or in a different pool. Since the question asks for ways to restrict a pod using labels, nodeName is an invalid answer.

Using spec.affinity.podAffinity with requiredDuringScheduling.

Why wrong: podAffinity with requiredDuringScheduling constrains a pod's placement relative to other pods, not relative to node labels. It uses pod labels and topology keys to ensure that the pod is scheduled onto nodes where certain other pods are running, which is about co-location or anti-affinity between workloads. This does not restrict a pod based on labels of the node itself, so it is not a correct answer here.

Create a ResourceQuota to limit the number of Pods per namespace.

Why wrong: A ResourceQuota is an administrative tool used to constrain aggregate resource consumption, such as CPU, memory, or total Pod count, within a specific namespace. It does not influence the Kubernetes scheduler's placement decisions or node selection algorithms. Consequently, it cannot force multiple Pod replicas to run on the same physical worker node.

Add a toleration for a taint that is present on only one node.

Why wrong: Taints and tolerations are designed to allow nodes to repel a set of Pods, not to attract them. While adding a toleration allows a Pod to be scheduled on a tainted node, it does not obligate the scheduler to place it there. The scheduler could still distribute the replicas across any of the other nine untainted worker nodes.

Add a nodeSelector to the Pod spec to select a specific node.

Why wrong: While a nodeSelector can restrict Pods to a specific node by matching node labels, hardcoding a specific node name or unique label reduces cluster resilience and creates a single point of failure. If that specific node becomes unhealthy, the scheduler cannot reschedule the Pods elsewhere. Furthermore, it does not dynamically colocate replicas on an arbitrarily chosen healthy node.

Set nodeName in the Pod spec to the node name.

Why wrong: Setting nodeName in the Pod spec directly binds the Pod to a specific node by its hostname. While this guarantees initial placement, it completely bypasses the scheduler and offers no resilience; if that particular node fails or is removed, the Pod will remain unschedulable until the node is restored, rather than being moved to another suitable node. This approach is too rigid for a critical application requiring flexibility and high availability.

Use nodeAffinity with requiredDuringSchedulingIgnoredDuringExecution and a matchExpression for disk=ssd.

Why wrong: A `requiredDuringSchedulingIgnoredDuringExecution` nodeAffinity enforces placement only at pod creation time, but if the labelled node becomes unavailable later, the scheduler will not move the pod to another node with `disk=ssd` — it merely keeps the pod on its current node. This approach is tempting because it guarantees initial scheduling on the correct node, which would be sufficient if the cluster never experienced node failures or maintenance.

Add a toleration for a taint that the node has.

Why wrong: Adding a toleration for a taint only allows a Pod to be scheduled onto a node that possesses that specific taint; it does not actively *select* or *force* the Pod onto any node, nor does it consider node labels like `disk=ssd`. Tolerations merely remove an exclusion barrier, permitting scheduling on tainted nodes, but they do not guarantee placement or specify desired node characteristics beyond the absence of blocking taints. Therefore, this mechanism cannot ensure the Pod runs on an SSD-equipped node.

The nodes do not have enough CPU or memory to run the Pod.

Why wrong: If the scheduling failure were caused by resource constraints, the kubectl describe pod output would explicitly show events with messages like FailedScheduling due to 'Insufficient cpu' or 'Insufficient memory'. However, the specific event output indicates that the pod is unschedulable due to node taints rather than resource exhaustion. Therefore, resource limits are not the root cause of this pending state.

The nodes are not tainted, but the Pod has tolerations.

Why wrong: Pods with tolerations can freely schedule onto untainted nodes as long as other scheduling criteria like resource requests and node selectors are met. Tolerations do not act as a requirement for taints to be present; they merely allow a pod to bypass existing taints. If the nodes were untainted, the pod would have scheduled successfully instead of remaining in a pending state with taint-related warnings.

The scheduler is not running.

Why wrong: If the kube-scheduler control plane component were completely offline, no scheduling decisions or evaluations would occur at all. Consequently, the pod would remain in a Pending state indefinitely without generating any FailedScheduling events in its event log. The presence of detailed scheduling failure events in the kubectl describe output proves that the scheduler is active and actively rejecting the pod based on taint constraints.