KCNA Cloud Native Observability — 20 Questions

Question 1

A DevOps team notices that a microservice is returning 503 errors intermittently. The service runs in Kubernetes and uses a liveness probe. The team wants to understand the root cause without restarting the pod. Which observability approach should they use first?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Query Prometheus for kubelet metrics on probe successes and failures. Prometheus can scrape kubelet metrics that expose liveness probe success and failure counts directly, allowing the team to see if the probe is failing without restarting the pod. This approach provides historical data on probe behavior, which is essential for diagnosing intermittent 503 errors that stem from the kubelet restarting the container when the liveness probe fails. Unlike other options, it does not require modifying the application or restarting the pod, and it directly surfaces the root cause if the probe is the issue.

Answer

Use kubectl describe pod to check recent events

Answer

Increase log verbosity in the application to capture all requests

Answer

Enable distributed tracing across the service mesh

Question 2

A platform team is designing a monitoring strategy for a multi-tenant Kubernetes cluster. Each tenant runs workloads in separate namespaces. The team needs to ensure tenant isolation while providing aggregated cluster-wide dashboards. Which approach best meets these requirements?

Accepted Answer

Deploy a Prometheus instance per tenant and use Thanos to aggregate metrics globally. Deploying a Prometheus instance per tenant enforces strong tenant isolation by preventing cross-tenant metric access or resource contention, while Thanos provides a global view by aggregating metrics from all tenants via sidecar-based or query-frontend federation. This approach satisfies both isolation and aggregated dashboards without compromising security or scalability.

Answer

Deploy a single Prometheus instance with namespace labels on all metrics

Answer

Use a global Prometheus with recording rules to aggregate per-namespace metrics

Answer

Have each tenant deploy their own monitoring stack and view separately

Question 3

A Kubernetes administrator is troubleshooting a pod that is stuck in CrashLoopBackOff. The pod's restart count is increasing. Which initial step should the administrator take to diagnose the issue?

Accepted Answer

Run 'kubectl logs ' to view the application logs. When a pod is in CrashLoopBackOff, the immediate priority is to inspect the application logs to understand why the container is failing. `kubectl logs ` retrieves the stdout/stderr output from the container, which typically contains error messages, stack traces, or configuration issues that caused the crash. This is the most direct and efficient first step before deeper investigation.

Answer

Run 'kubectl describe pod <pod-name>' to check events

Answer

Check the Prometheus metrics for the pod's CPU usage

Answer

Run 'kubectl exec -it <pod-name> -- /bin/sh' to inspect the container

Question 4

An organization uses Prometheus and Grafana for monitoring. They want to alert when the 99th percentile of request latency exceeds 500ms for more than 5 minutes. Which PromQL query should they use in the alert rule?

Accepted Answer

histogram_quantile(0.99, rate(http_request_duration_seconds_bucket[5m])) > 0.5. It uses `histogram_quantile(0.99, rate(...[5m]))` to calculate the 99th percentile request latency over a 5-minute window, matching the requirement to alert when this value exceeds 500ms (0.5 seconds) for more than 5 minutes. The `rate()` function with a 5m range computes the per-second increase of bucket counters over that duration, which is necessary for accurate quantile estimation in Prometheus.

Answer

histogram_quantile(0.99, rate(http_request_duration_seconds_bucket[1m])) > 0.5

Answer

avg(rate(http_request_duration_seconds_bucket[5m])) > 0.5

Answer

max(rate(http_request_duration_seconds_bucket[5m])) > 0.5

Question 5

A company runs a Kubernetes cluster with 50 worker nodes, each hosting multiple microservices. They use Prometheus for metrics collection and Grafana for dashboards. Recently, the Prometheus server has been experiencing out-of-memory (OOM) kills during peak hours, causing gaps in metric collection. The cluster has a dedicated monitoring namespace. The team has already increased the Prometheus pod's memory limits to 8GB, but OOMs still occur. The metrics retention is set to 15 days. The cardinality of certain metrics (e.g., HTTP request labels with user IDs) is very high. The team needs to resolve the OOM issue without losing critical alerting capability for at least the last 7 days of data. Which action should they take first?

Accepted Answer

Implement recording rules to pre-aggregate high-cardinality metrics at a lower granularity. Recording rules allow Prometheus to pre-aggregate high-cardinality metrics (e.g., HTTP request labels with user IDs) at a lower granularity, reducing the number of unique time series stored in memory. This directly addresses the OOM issue caused by cardinality explosion without discarding raw data entirely, preserving the ability to query aggregated metrics for alerting over the required 7-day window.

Answer

Drop high-cardinality metrics like HTTP request labels using relabel_configs

Answer

Reduce metrics retention to 7 days to free memory

Answer

Enable vertical pod autoscaler for the Prometheus pod

Question 6

A company deploys a microservice application on Kubernetes. They notice that one of the services is returning 5xx errors intermittently. Which observability tool should they use to correlate the errors with resource usage across all pods of that service?

Accepted Answer

Prometheus. Prometheus is the correct choice because it is a monitoring and alerting toolkit designed to collect and store time-series metrics, such as CPU, memory, and request error rates. By querying Prometheus with PromQL, you can correlate 5xx error spikes with resource usage across all pods of a service, as it scrapes metrics from each pod's /metrics endpoint. This direct correlation of application-level errors with infrastructure metrics is not natively provided by the other tools listed.

Answer

Grafana

Answer

Fluentd

Answer

Jaeger

Question 7

You are an SRE managing a Kubernetes cluster with 200 nodes and 10,000 pods. The cluster runs a critical payment processing application. Users report that transactions are occasionally failing with a 'timeout' error. You have Prometheus and Grafana set up for monitoring, and you use Fluentd with Elasticsearch for logging. You notice that during peak hours, the CPU usage of the payment service pods spikes to 90%, but memory usage remains stable. The pod restart count is low. You also see that the response time of the payment service increases significantly during these spikes. You need to identify the root cause and propose a fix. Which course of action is most appropriate?

Accepted Answer

Increase the CPU limits for the payment service pods to allow more CPU resources during spikes. The CPU usage spikes to 90% during peak hours, indicating that the payment service pods are CPU-bound. Increasing CPU limits allows the pods to burst and utilize more CPU resources, reducing response times and preventing timeouts. This directly addresses the bottleneck without adding unnecessary replicas or changing memory settings.

Answer

Add more replicas of the payment service to distribute the load

Answer

Increase the memory limits for the payment service pods to improve caching

Answer

Implement a circuit breaker pattern to fail fast and avoid timeouts

Question 8

Match each Kubernetes security concept to its definition.

Accepted Answer

RBAC: Controls access to the Kubernetes API based on roles and permissions.. RBAC and NetworkPolicy are fundamental security mechanisms. ServiceAccount provides identity. The distractors confuse RBAC with secrets management and NetworkPolicy with user access control.

Answer

RBAC: Manages secrets and encryption at rest.

Answer

NetworkPolicy: Controls which users can access the cluster.

Question 9

Which command retrieves logs from a specific container named 'sidecar' in a multi-container pod?

Accepted Answer

kubectl logs pod-name -c sidecar. The -c flag specifies the container name.

Answer

kubectl logs pod-name sidecar

Answer

kubectl logs pod-name --container sidecar

Answer

kubectl logs sidecar pod-name

Question 10

Which TWO are components of a distributed trace? (Select two.)

Accepted Answer

Spans. A distributed trace is composed of spans, which represent individual units of work within the trace. The trace itself is the overarching collection of spans, not a component. Metrics, logs, and alerts are separate observability signals and not components of a trace.

Answer

Metrics

Answer

Traces

Answer

Logs

Answer

Alerts

Question 11

A developer wants to view the logs of a specific container named 'sidecar' in a pod called 'web-app'. Which command should they use?

Accepted Answer

kubectl logs web-app -c sidecar. The -c flag specifies the container name when a pod has multiple containers.

Answer

kubectl logs sidecar --pod web-app

Answer

kubectl logs -c sidecar web-app

Answer

kubectl logs sidecar web-app

Question 12

A company uses Prometheus for monitoring and wants to alert when the average CPU usage over 5 minutes exceeds 80%. Which PromQL query would correctly define this alert rule?

Accepted Answer

avg(rate(node_cpu_seconds_total{mode!="idle"}[5m])) > 0.8. The correct query uses rate() over a 5-minute window on the non-idle CPU counter, then averages across all CPU series. node_cpu_seconds_total is a counter, so you must apply rate() to get per-second usage; filtering mode!="idle" isolates busy CPU time, and avg() aggregates across cores and instances. Comparing the resulting fraction to 0.8 expresses 80% utilization.

Answer

avg(rate(node_cpu_seconds_total{mode="idle"}[5m])) > 0.8

Answer

avg(node_cpu_seconds_total{mode!="idle"}[5m]) > 0.8

Answer

sum(rate(node_cpu_seconds_total{mode!="idle"}[5m])) > 0.8

Question 13

A team wants to set up alerts when a Kubernetes pod consumes more than 90% of its memory limit for over 5 minutes. They use Prometheus and Alertmanager. Which Prometheus query would trigger an alert for a specific pod named 'web-app' in the 'default' namespace?

Accepted Answer

container_memory_usage_bytes{pod='web-app'} / container_spec_memory_limit_bytes{pod='web-app'} > 0.9. It calculates the ratio of actual memory usage to the configured memory limit for the pod, then checks if that ratio exceeds 0.9 (90%). This directly answers the requirement to alert when a pod consumes more than 90% of its memory limit. The query uses two Prometheus metrics: container_memory_usage_bytes (current usage) and container_spec_memory_limit_bytes (the limit), both filtered by pod name. Dividing them yields a dimensionless fraction that can be compared to 0.9.

Answer

container_memory_usage_bytes{pod='web-app'} > 0.9

Answer

avg(container_memory_usage_bytes{pod='web-app'}) > 0.9

Answer

container_memory_limit_bytes{pod='web-app'} > 0.9

Question 14

Which TWO of the following are common log aggregation tools used in Kubernetes environments? (Select two)

Accepted Answer

Fluentd. Fluentd (B) is correct because it is a widely deployed open-source log aggregator and unified logging layer in Kubernetes, commonly run as a DaemonSet to collect container logs from /var/log/containers and forward them to backends such as Elasticsearch or Loki. Fluent Bit (E) is correct because it is a lightweight, C-based log processor and forwarder designed for resource-constrained environments, frequently used in Kubernetes as a DaemonSet or sidecar to collect and ship container logs. Loki (A) is a log aggregation system, but it is a storage/query backend rather than a log collection and forwarding agent, so it is not the aggregation tool being asked about here. Prometheus (C) is a metrics monitoring and time-series database system, not a log aggregation tool. Jaeger (D) is a distributed tracing system for request tracing, not log aggregation.

Answer

Loki

Answer

Prometheus

Answer

Jaeger

Question 15

What is the primary purpose of structured logging?

Accepted Answer

To enable automated analysis and querying of logs. Structured logging formats log data in a consistent, machine-parseable format (e.g., JSON) with key-value pairs. This enables automated tools like Elasticsearch, Loki, or Splunk to efficiently index, search, filter, and aggregate logs, which is essential for observability at scale. The primary purpose is to facilitate automated analysis and querying, not to replace other telemetry signals or to focus on human readability alone.

Answer

To replace metrics and traces

Answer

To reduce the size of log files

Answer

To make logs human-readable only

Question 16

Which THREE of the following are benefits of using a service mesh for observability? (Select three.)

Accepted Answer

Distributed tracing across services. Option C (Distributed tracing across services) is correct because a service mesh's sidecar proxies can propagate trace context and emit span data for every request traversing the mesh, giving end-to-end visibility across service boundaries without changing application code. Option D (Collection of detailed metrics for service-to-service communication) is correct because sidecars capture L7 telemetry such as request rate, error rate, latency histograms, and TCP connection stats for each hop, which is exactly the kind of granular inter-service metric a mesh exposes. Option E (Automatic instrumentation of application code for traces) is correct because the mesh injects sidecar proxies that transparently instrument traffic, so traces are generated without developers manually adding tracing libraries to each service. Option A (Reduced network latency) is not a benefit of observability—sidecar interception typically adds a small amount of latency rather than reducing it, and latency reduction is a traffic-management concern, not observability. Option B (Centralized logging of all application logs) is not provided by a service mesh; meshes emit access/telemetry data, but aggregating application log output is the job of a logging pipeline (e.g., Fluentd/ELK), not the mesh itself.

Answer

Reduced network latency

Answer

Centralized logging of all application logs

Question 17

Which TWO of the following are valid components of the Alertmanager configuration? (Select two.)

Accepted Answer

route. Alertmanager configuration includes 'route' for routing alerts and 'receivers' for notification channels. 'prometheus_rules' is part of Prometheus configuration, not Alertmanager. 'alert' and 'group_by' are not top-level Alertmanager config keys.

Answer

group_by

Answer

prometheus_rules

Answer

alert

Question 18

Which THREE of the following are core components of the OpenTelemetry specification? (Select three.)

Accepted Answer

Data Model. The OpenTelemetry specification defines three core components: the Data Model (A), which standardizes signals like traces, metrics, and logs along with their attributes and semantics; the API (B), which provides vendor-neutral interfaces for instrumenting code to generate telemetry; and the SDK (E), which implements the API with configurable processors, samplers, and exporters for actually producing and emitting telemetry. The Collector (C) and Exporter (D) are not core components of the specification itself — the Collector is a separate, standalone service (part of the OpenTelemetry project but not the spec), and exporters are implementation details within the SDK rather than a top-level specification component.

Answer

Collector

Answer

Exporter

Question 19

A team is deploying a new microservice on Kubernetes and wants to implement the three pillars of observability. They plan to use OpenTelemetry for instrumentation. Which TWO of the following are considered pillars of observability that OpenTelemetry can help collect? (Choose two.)

Accepted Answer

Metrics. The three pillars of observability are metrics, logs, and traces. OpenTelemetry provides instrumentation for all three, but in many KCNA contexts, metrics and traces are highlighted as the primary signals it collects. Configuration maps and container images are not observability signals. The two correct pillars here are metrics and traces.

Answer

Logs

Answer

Configuration maps

Answer

Container images

Question 20

Which TWO of the following are best practices for structuring log output in cloud-native applications to maximize observability?

Accepted Answer

Output logs in structured format such as JSON. Option C is correct because emitting logs as structured data such as JSON lets log processors and observability backends parse fields programmatically, enabling reliable filtering, aggregation, and indexing by attributes like severity, service, and trace context rather than relying on fragile regex over free text. Option D is correct because including a unique request or correlation ID (for example a W3C traceparent trace ID or an application-generated correlation ID) in every entry allows distributed traces and logs to be stitched together across microservices, which is essential for root-cause analysis in cloud-native systems. The remaining options are not best practices: A floods storage and cost with low-value debug noise and can obscure real signals, B breaks line-oriented log collectors and parsers that expect one event per line, and E removes timestamps that are required to order events and correlate them across services and time zones.

Answer

Include verbose debug-level information in every log line

Answer

Use multi-line log entries for detailed error information

Answer

Avoid timestamps to reduce log size