Cloud Native Observability practice questions

Traces

Why wrong: This is the entire trace, not a component of it. The question asks for components, so 'Traces' is incorrect.

Log

Why wrong: Log is not a Prometheus metric type.

Event

Why wrong: Event is not a Prometheus metric type.

Trace

Why wrong: Trace is not a Prometheus metric type.

Use kubectl describe pod to check recent events

Why wrong: Events may not capture every probe failure, especially if they are short-lived.

Increase log verbosity in the application to capture all requests

Why wrong: Logs may not capture probe failures, and increasing verbosity can impact performance.

Enable distributed tracing across the service mesh

Why wrong: Tracing is for request flows, not probe health checks.

Deploy a single Prometheus instance with namespace labels on all metrics

Why wrong: Single Prometheus does not enforce access control; any user could query metrics across namespaces.

Use a global Prometheus with recording rules to aggregate per-namespace metrics

Why wrong: This does not provide isolation; all metrics are accessible in one place.

Have each tenant deploy their own monitoring stack and view separately

Why wrong: Lacks aggregated cluster-wide dashboards, violating the requirement.

Run 'kubectl describe pod <pod-name>' to check events

Why wrong: Events may indicate issues but usually lack application-level error details.

Check the Prometheus metrics for the pod's CPU usage

Why wrong: CPU metrics are unlikely to reveal crash causes.

Run 'kubectl exec -it <pod-name> -- /bin/sh' to inspect the container

Why wrong: Cannot exec into a crashed pod; it must be running.

histogram_quantile(0.99, rate(http_request_duration_seconds_bucket[1m])) > 0.5

Why wrong: Range of 1m is too short; need 5m to match requirement.

avg(rate(http_request_duration_seconds_bucket[5m])) > 0.5

Why wrong: This averages bucket rates, not percentile.

max(rate(http_request_duration_seconds_bucket[5m])) > 0.5

Why wrong: Max of bucket rates is not percentile.

Include verbose debug-level information in every log line

Why wrong: Too much verbosity increases storage and noise; use appropriate levels.

Use multi-line log entries for detailed error information

Why wrong: Multi-line logs are harder to parse; use single-line with escaped newlines if needed.

Avoid timestamps to reduce log size

Why wrong: Timestamps are essential for ordering and debugging.

Monitoring CPU and memory usage of each service instance

Why wrong: Resource metrics are better for Prometheus.

Capturing detailed error messages and stack traces

Why wrong: Distributed tracing focuses on propagating trace context across service boundaries to reconstruct end-to-end request flows, not on capturing the payload of individual errors. Detailed error messages and stack traces belong to structured logging or exception-handling frameworks, which record local failure details without the cross-service correlation identifiers that tracing requires. This option is tempting because observability tools often bundle tracing with logging, but the specific mechanism of trace propagation—span IDs and parent-child relationships—is absent from raw error capture.

Drop high-cardinality metrics like HTTP request labels using relabel_configs

Why wrong: This may remove useful metrics; better to aggregate them.

Reduce metrics retention to 7 days to free memory

Why wrong: Reduces memory but also loses older data; less targeted than recording rules.

Enable vertical pod autoscaler for the Prometheus pod

Why wrong: VPA adjusts resources but does not reduce cardinality; OOM may still occur if node is saturated.

Grafana

Why wrong: Grafana visualizes data from sources like Prometheus, but does not store or correlate metrics itself.

Fluentd

Why wrong: Fluentd is a log collector, not a metrics system.

Jaeger

Why wrong: Jaeger is for distributed tracing, not metric correlation.

Increase log retention to one year for compliance

Why wrong: Increases cost without addressing the need to reduce storage costs.

Store all logs in a centralized Elasticsearch cluster with high retention

Why wrong: Storing all logs is costly; not the best cost optimization.

Aggregate logs into a single pod for easier indexing

Why wrong: Single pod is a single point of failure and doesn't reduce cost.

kube_deployment_spec_replicas

Why wrong: This shows desired replicas, but alone doesn't show health.

kube_node_status_condition

Why wrong: Node-level metric, not deployment health.

kube_pod_container_status_running

Why wrong: This is a pod-level metric, not deployment-level.

Store all raw observability data indefinitely for forensic analysis

Why wrong: Storing all data indefinitely is costly and not a best practice.

Use only metrics and avoid logs to reduce complexity

Why wrong: Metrics alone lack context; logs and traces are also needed.

Randomly sample all traces and logs to reduce storage

Why wrong: Random sampling may miss critical errors; targeted sampling is better.

Add more replicas of the payment service to distribute the load

Why wrong: Horizontal scaling helps but may not address the root cause if the service is inefficient; also adds complexity.

Increase the memory limits for the payment service pods to improve caching

Why wrong: Memory is stable; increasing memory won't help CPU-bound issues.

Implement a circuit breaker pattern to fail fast and avoid timeouts

Why wrong: Circuit breaker prevents cascading failures but doesn't fix the underlying CPU issue.

Monitor CPU and memory metrics from kube-state-metrics and correlate with latency.

Why wrong: Resource metrics may not directly indicate request latency.

Increase log verbosity for all services and search for error messages.

Why wrong: Logs may not capture latency across services.

Check node-level metrics using Prometheus Node Exporter.

Why wrong: Node metrics are too granular for service-level latency.

Use a single centralized logging solution to aggregate logs from all components.

Why wrong: Single point of failure; prefer highly available solutions.

Store all debug logs for a minimum of 90 days for compliance.

Why wrong: Debug logs are verbose; store selectively.

Disable leader election for monitoring components to reduce complexity.

Why wrong: Leader election prevents duplicate metrics in HA setups.

Save snapshot, Verify, Access the node, Store securely

Why wrong: This order is incorrect because you cannot save a snapshot without first accessing the node. Verification also requires the snapshot file to exist.

Access the node, Store securely, Save snapshot, Verify

Why wrong: This order is incorrect because you must save the snapshot before storing it securely, and verification should occur after saving but before storage.

Verify, Access the node, Save snapshot, Store securely

Why wrong: This order is incorrect because verification requires an existing snapshot, so it cannot be the first step. Access the node must come first.

RBAC: Manages secrets and encryption at rest.

Why wrong: Incorrect — managing secrets and encryption is handled by Secrets and encryption configuration, not RBAC.

NetworkPolicy: Controls which users can access the cluster.

Why wrong: Incorrect — user access control is handled by authentication and RBAC, not network policies.

Alerting

Why wrong: Alerting is not a pillar; it is a downstream action from metrics.

SLIs

Why wrong: SLIs are service level indicators, not a pillar.

Dashboards

Why wrong: Dashboards are visualizations, not a pillar.

Provide distributed tracing

Why wrong: Tracing is handled by Jaeger or Zipkin, not Prometheus.

Visualize data

Why wrong: Visualization is typically done with Grafana, not Prometheus.

Collect and store logs

Why wrong: Log collection is not Prometheus's primary function.