CCNA Cloud Native Observability Questions

Option C is correct because the OpenTelemetry Collector is a vendor-agnostic proxy that receives telemetry data (traces, metrics, logs) from instrumented applications, processes it (e.g., batching, filtering, enrichment), and exports it to one or more backends (e.g., Jaeger, Prometheus, or any OTLP-compatible system). It decouples data generation from data storage, enabling flexible, scalable observability pipelines without vendor lock-in.

rate() and histogram_quantile() are common PromQL functions. avg_over_time() is also valid but avg is not a function, it's an aggregation operator.

The -c flag specifies the container name when a pod has multiple containers.

A gauge is a metric that can go up and down, like memory usage or temperature.

OpenTelemetry includes specification, SDKs, Collector, and instrumentations. Prometheus and Jaeger are separate projects.

OpenTelemetry provides a unified standard that avoids vendor lock-in, allowing data to be sent to any backend.

A counter is a cumulative metric that only increases (or resets to zero). It is ideal for counting events like HTTP requests.

Option B is correct because it uses `histogram_quantile(0.99, rate(...[5m]))` to calculate the 99th percentile request latency over a 5-minute window, matching the requirement to alert when this value exceeds 500ms (0.5 seconds) for more than 5 minutes. The `rate()` function with a 5m range computes the per-second increase of bucket counters over that duration, which is necessary for accurate quantile estimation in Prometheus.

The metrics-server provides resource metrics (CPU and memory) per pod and node, used by kubectl top and the Horizontal Pod Autoscaler.

Prometheus supports counter, gauge, histogram, and summary. Options A and D are valid; B (timer) and C (quantile) are not; E (meter) is from OpenTelemetry.

The OpenTelemetry Collector is a vendor-agnostic agent that receives, processes, and exports telemetry data.

Alertmanager handles alerts sent by Prometheus server, deduplicates, groups, and routes them to receivers (email, Slack, etc.), and manages silencing and inhibition.

Access the node, save snapshot, verify, store securely, and restore when necessary.

The three pillars of observability are logs, metrics, and traces. Alerting is derived from metrics, not a pillar itself.

A counter is a cumulative metric that only increases, ideal for counting requests.

A service mesh provides visibility into inter-service communication, adds distributed tracing without code changes, and collects metrics like request latency.

Option B is correct because Prometheus can scrape kubelet metrics that expose liveness probe success and failure counts directly, allowing the team to see if the probe is failing without restarting the pod. This approach provides historical data on probe behavior, which is essential for diagnosing intermittent 503 errors that stem from the kubelet restarting the container when the liveness probe fails. Unlike other options, it does not require modifying the application or restarting the pod, and it directly surfaces the root cause if the probe is the issue.

The three pillars are logs, metrics, and traces. Alerts are derived from these pillars but not considered a pillar themselves.

Incomplete traces often occur when context propagation is not implemented correctly, causing spans to be disconnected.

A counter is a cumulative metric that can only increase or be reset to zero, ideal for counting events like requests.

Loki is a log aggregation system optimized for Kubernetes, designed to be cost-effective and easy to operate.

Prometheus metric types include Counter, Gauge, Histogram, and Summary. Options A and B are correct.

The correct query divides container memory usage by its limit and compares to 0.9.

The three pillars of observability in cloud native environments are logs, metrics, and traces.

Fluent Bit is a lightweight log processor and forwarder, often used in Kubernetes.

Prometheus is a metric system that collects and stores numeric time-series data.

SLO of 99.9% means up to 0.1% errors are allowed. 0.1% of 10,000,000 is 10,000 requests.

The OpenTelemetry specification defines the API, SDK, and data model. These are the core components.

Option D is correct because structured logging (e.g., JSON format) enables efficient indexing and querying of logs, while sampling debug logs and retaining error logs fully reduces storage costs without losing critical error patterns. This approach balances observability needs with cost optimization, a key principle in cloud-native environments.

Alertmanager configuration includes 'route' for routing alerts and 'receivers' for notification channels. 'prometheus_rules' is part of Prometheus configuration, not Alertmanager. 'alert' and 'group_by' are not top-level Alertmanager config keys.

Fluent Bit is a lightweight log processor and forwarder, designed for resource-constrained environments like Kubernetes nodes.

Grafana is the most popular visualization tool for Prometheus metrics, offering rich dashboards.

Option A is correct because cloud provider billing APIs provide actual cost data per resource (e.g., per node, per persistent volume, per network egress), and combining this with resource utilization data (e.g., CPU/memory requests and actual usage from metrics) enables accurate cost allocation per namespace, pod, or workload. This approach directly maps infrastructure spend to Kubernetes abstractions, which is essential for chargeback or showback in multi-tenant clusters.

Jaeger is a CNCF graduated project focused on distributed tracing.

The OpenTelemetry Collector receives, processes, and exports telemetry data.

Jaeger and Zipkin are popular open-source distributed tracing systems.

The rate() function calculates the per-second average rate of increase of a counter over a time range.

The OpenTelemetry Collector is a vendor-agnostic agent or gateway that receives telemetry data (traces, metrics, logs) from instrumented applications, processes it (e.g., batching, filtering, sampling), and exports it to one or more backends (e.g., Jaeger, Prometheus, or any OTLP-compatible system). It decouples data generation from data export, enabling flexible pipeline management without modifying application code.

Option D is correct because when a pod is in CrashLoopBackOff, the immediate priority is to inspect the application logs to understand why the container is failing. `kubectl logs <pod-name>` retrieves the stdout/stderr output from the container, which typically contains error messages, stack traces, or configuration issues that caused the crash. This is the most direct and efficient first step before deeper investigation.

An SLI is a specific metric that measures a particular aspect of service reliability, such as request latency or error rate.

Option C is correct because including correlation IDs in structured logs is a key observability practice that enables distributed tracing across microservices. In Kubernetes, where requests often traverse multiple pods and services, correlation IDs allow you to link logs from different components into a single transaction flow, which is essential for debugging and understanding system behavior.

A span represents a unit of work in a distributed system, often a single operation like an HTTP request or database call.

The metrics-server is the correct component because it is specifically designed to collect resource metrics (CPU and memory) from the kubelet on each node via the Summary API and expose them through the Kubernetes Metrics API. This allows tools like `kubectl top` and the Horizontal Pod Autoscaler to access real-time resource usage without requiring a full monitoring stack.

The three pillars are logs, metrics, and traces. Security is not one of them, though it is important.

SLOs should be based on SLIs, define a target (e.g., 99.9%), and include a measurement window. Cost is not a direct consideration for SLO definition.

A span is the fundamental building block of a trace, representing a single unit of work in a distributed system.

These are key security mechanisms in Kubernetes.

Structured logging formats log data in a consistent, machine-parseable format (e.g., JSON) with key-value pairs. This enables automated tools like Elasticsearch, Loki, or Splunk to efficiently index, search, filter, and aggregate logs, which is essential for observability at scale. The primary purpose is to facilitate automated analysis and querying, not to replace other telemetry signals or to focus on human readability alone.

Fluentd and Fluent Bit are both popular log aggregators and forwarders in Kubernetes. Loki is a log storage system, not an aggregator.

Context propagation carries trace context across service boundaries to connect spans into a single trace.

Structured logging outputs logs in a consistent, machine-readable format (e.g., JSON) making it easier to parse, filter, and analyze log data.

Jaeger is a popular open-source distributed tracing system.

Structured logging provides machine-parseable output, enables easier querying and analysis, and ensures consistent field naming. Human readability is not a primary benefit; structured logs can be less human-friendly than plain text.

Alertmanager is the component in the Prometheus ecosystem responsible for handling alerts fired by the Prometheus server. It deduplicates, groups, and routes alerts to configured notification channels such as email, PagerDuty, or Slack, ensuring that operators receive actionable notifications without alert fatigue.

OpenTelemetry provides SDKs for instrumentation, a collector for processing, and APIs for standards.

Option C is correct because adding unique request IDs (e.g., via OpenTelemetry trace IDs or custom correlation IDs) to logs enables end-to-end tracing across microservices. This allows operators to correlate a single user request as it traverses multiple services, which is essential for debugging distributed systems in a cloud-native environment.

Option A is correct because `kube_deployment_status_replicas_ready` directly exposes the number of ready replicas for a Deployment, which can be compared against `kube_deployment_spec_replicas` to determine if the desired state matches the actual healthy state. This metric is emitted by kube-state-metrics, which generates Prometheus-compatible metrics from Kubernetes API objects, making it the standard choice for monitoring Deployment health.

An SLO is a target level of reliability, expressed as a percentage of a metric over a time window.

The query should calculate the average CPU usage rate over 5 minutes and compare it to 0.8 (80%).

A trace consists of spans, which are individual units of work.

Prometheus pulls metrics from targets at regular intervals, which is the pull-based model.

Alertmanager uses routes to match alerts and receivers to send notifications. Routes define grouping and routing logic.

Prometheus is the correct choice because it is a monitoring and alerting toolkit designed to collect and store time-series metrics, such as CPU, memory, and request error rates. By querying Prometheus with PromQL, you can correlate 5xx error spikes with resource usage across all pods of a service, as it scrapes metrics from each pod's /metrics endpoint. This direct correlation of application-level errors with infrastructure metrics is not natively provided by the other tools listed.

Context propagation allows trace context to be passed between services, enabling distributed tracing correlation.

kubectl logs fetches the standard output and standard error logs from a container in a pod.

Fluent Bit is a lightweight log processor and forwarder, often used as a DaemonSet to collect logs.

Option D is correct because the CPU usage spikes to 90% during peak hours, indicating that the payment service pods are CPU-bound. Increasing CPU limits allows the pods to burst and utilize more CPU resources, reducing response times and preventing timeouts. This directly addresses the bottleneck without adding unnecessary replicas or changing memory settings.

Logs and Metrics are two of the three pillars of observability (alongside Traces). Logs provide immutable, timestamped records of discrete events, while Metrics are numeric aggregations of data over time (e.g., Prometheus counters, histograms). Together they form the foundation for understanding system behavior in cloud-native environments.

OpenTelemetry (OTel) is the industry standard for observability data collection and export, providing vendor-agnostic instrumentation.

Distributed tracing is designed to track the flow of a single request across multiple microservices, recording timing and causality. Option B is correct because tracing systems like Jaeger or Zipkin automatically build a dependency graph by analyzing the parent-child relationships between spans, which reveals how services interact. This is a core use case for understanding service topology and identifying bottlenecks in a distributed system.

The -c flag specifies the container name.

SLA stands for Service Level Agreement, a contract specifying expected service level.

Option C is correct because structured logging (e.g., JSON) enables automated parsing, filtering, and querying by log aggregation tools like Fluentd, Logstash, or cloud-native observability backends (e.g., Elasticsearch, Loki). This format ensures each log entry has consistent key-value pairs, making it machine-readable and facilitating correlation across distributed services without manual text parsing.

The -c flag specifies the container name. The correct command is 'kubectl logs app-pod -c sidecar'.

Option D is correct because deploying a Prometheus instance per tenant enforces strong tenant isolation by preventing cross-tenant metric access or resource contention, while Thanos provides a global view by aggregating metrics from all tenants via sidecar-based or query-frontend federation. This approach satisfies both isolation and aggregated dashboards without compromising security or scalability.