Prometheus Certified Associate (PCA, CNCF/Linux Foundation) (PCA) (PCA) — Questions 1–75

In a pull model Prometheus initiates each scrape, so a failed connection or timeout directly signals that the target is down. This lets the server detect target failure itself, rather than waiting for the target to push data, satisfying the scenario's requirement for the primary pull-based benefit.

node_exporter's default listen address is :9100, so it serves metrics on TCP port 9100 unless overridden with --web.listen-address. This default port is what Prometheus scrape configs for node targets conventionally reference, satisfying the question's requirement for a true statement.

kubernetes_sd_configs is the service discovery block that queries the Kubernetes API for pods, services and endpoints. It supplies the role-based targets Prometheus needs to find pods labelled app=frontend, satisfying the discovery requirement in the stem.

Labels are key-value pairs attached to a metric name; the full set of label values uniquely identifies each time series within Prometheus's TSDB. This satisfies the stem's purpose, distinguishing individual series that share the same metric name.

Recording rules precompute expensive expressions into new series, so dashboards and alerts reference a single stored metric instead of re-running complex PromQL. This satisfies the stem's benefit by simplifying complex queries and reducing repeated evaluation cost at query time.

`group_by` aggregates alerts sharing specified label values into a single notification, directly satisfying the stem's requirement to group alerts. It defines which labels Alertmanager uses to batch related alerts, reducing notification noise. Other labels merely route or match alerts rather than controlling aggregation.

Adding a Prometheus data source in Grafana queries the Prometheus HTTP API directly, so no intermediary exporter or federation layer is needed. This satisfies the stem's requirement to visualise existing Prometheus metrics, since Grafana's built-in data source plugin translates PromQL queries and renders the returned time series natively.

The Prometheus server performs the scraping: it pulls metrics from configured targets at set intervals via HTTP. For a custom exporter, the exporter merely exposes metrics in Prometheus format on an endpoint; the server initiates collection. This satisfies the stem's requirement by identifying the active puller rather than the passive endpoint.

Client libraries accept a label dictionary (map of label names to values) passed at metric creation or observation time, which attaches those key-value pairs to the sample. This is the mechanism for adding dimensions such as method or status to a custom metric.

Storage is a resource metric, not one of the four Golden Signals: latency, traffic, errors and saturation. Those four, from Google's SRE guidance, describe user-facing service health, whereas storage capacity is an underlying infrastructure concern.

Binary operators match series by their full label sets. When A and B share no identical label combination, no pairing exists, so the vector matching yields nothing and the expression returns an empty result rather than an error.

irate() computes a per-second rate from only the last two samples in the range, making it responsive to sudden spikes and suitable for volatile counter metrics. This satisfies the scenario's requirement to calculate rates or increases from counters, unlike functions needing longer windows.

The `=` matcher filters time series where the label value equals the given string exactly, including case sensitivity. This satisfies the stem's requirement for a correct label matcher statement, as PromQL's equality operator performs full-string comparison rather than substring or regex matching.

Raising evaluation_interval lengthens the gap between rule evaluations, so each rule query executes less often and consumes proportionally less CPU. Recorded series remain written to storage, satisfying the constraint of keeping data available while cutting load.

A scrape fails when the target returns a non-2xx status, when the connection times out, or when the exposition body cannot be parsed. Invalid metric format breaks parsing, so the scrape is marked unsuccessful even though the HTTP request itself succeeded.

Alertmanager validates configuration on reload; if parsing fails, it rejects the new file and keeps the previously loaded configuration running. Alerts therefore continue routing normally, satisfying the stem's scenario without dropping notifications or halting the service.

Each unique label combination creates a distinct time series held in memory, so ephemeral pods with churning names, IDs or hashes generate vast numbers of short-lived series. This memory overhead, not scrape volume, is the dominant cardinality constraint in dynamic clusters.

A one-minute range spans only a few samples at typical 15–60 second scrape intervals, so PromQL's rate() extrapolation amplifies scrape jitter and produces volatile, unreliable per-second values. This directly addresses the stem's constraint: short windows relative to scrape intervals yield noisy, jitter-sensitive results unsuitable for production alerting.

avg_over_time computes the mean of all samples within the specified range, so wrapping node_load1 in [10m] yields a rolling ten-minute average that damps short spikes. This directly satisfies the stem's requirement to smooth short-term fluctuations in a gauge.

Variables enable users to dynamically switch or select data sources in a panel.

Latency as a Golden Signal distinguishes successful-request duration from error duration, so you can break down response time by dependency. If external calls dominate the slow percentile, the latency originates outside the service, directly satisfying the need to isolate external-dependency causation.

The ignoring modifier removes the named labels from the matching signature before equality is tested. Excluding instance and region leaves job as the sole common label, so samples pair on job alone despite differing label sets, satisfying the join requirement.

An instant vector holds one sample per series at a single evaluation timestamp, whereas a range vector holds every sample within a specified lookback window for each series. This satisfies the stem's distinction, and only range vectors can feed functions such as rate() or increase().

`count` aggregates across series, yielding one output series per input group, whereas `count_values` buckets by observed sample value and emits one series per distinct value, exposing its frequency. This satisfies the stem's requirement to distinguish cardinality counting from value-frequency distribution.

The Prometheus HTTP API exposes the `/api/v1/rules` endpoint, which returns currently configured alerting and recording rules, optionally filtered by type, rule name or group. This directly satisfies the stem's requirement to retrieve alert rules, distinguishing it from endpoints handling queries, targets or metadata.

label_replace() applies the supplied regex to the source label's value, and on a match writes the replacement string, with capture-group references, into the destination label. It rewrites labels for grouping or joining; it does not alter sample values or timestamps.

Collecting every available metric floods dashboards and alerts with low-value data, obscuring genuine incidents. It also inflates storage, scrape load and licensing costs. Avoiding this preserves signal clarity and controls resource expenditure, satisfying the stem's rationale.

Time series panels plot metric values against a timestamped x-axis, rendering successive data points as a continuous line. This directly satisfies the requirement to visualise how a metric changes over time, unlike stat, gauge or table panels which show only current or aggregated values.

Histograms bucket observations into configurable ranges and expose cumulative counts plus a sum, letting you calculate quantiles such as p95 latency across aggregated series. That distributional view is what latency analysis needs, unlike a gauge or counter, satisfying the requirement to capture request latency distribution.

The http_2xx module probes the endpoint and marks success only when the returned HTTP status code falls in the 2xx range, directly satisfying the requirement to check the status code. Other modules such as tcp_connect or icmp test reachability, not HTTP response codes.

`unless` returns left-hand samples only where no matching right-hand series exists, filtering by label set. It satisfies the stem's requirement for a valid PromQL set operator combining instant vectors, alongside `and` and `or`. Unlike arithmetic binary operators, it operates purely on label matching rather than values.

`predict_linear` fits a least-squares linear regression over the 1h range vector and extrapolates it 4 hours ahead, satisfying the stem's demand for predictable linear extrapolation of a gauge over time. It returns the projected value of `node_filesystem_free_bytes`, exposing when the filesystem will exhaust free space.

dns_sd_configs performs DNS SRV or A record lookups to discover targets, refreshing them on a configurable interval. This satisfies the stem's valid discovery mechanism, since Prometheus supports it natively alongside static_configs, file_sd_configs and the cloud provider integrations.

Each metric sample occupies its own line, with the metric name, optional labels and value separated by whitespace, and lines terminated by a line feed. This line-oriented structure is what parsers rely on, satisfying the requirement for a true statement about the exposition format.

Prometheus Pushgateway lets the short-lived batch job push its metrics before exiting, since Prometheus's pull model cannot scrape a process that only exists for 30 seconds every 24 hours. The Pushgateway holds those metrics until Prometheus scrapes it, satisfying the ephemeral-job constraint.

Client-side quantiles are computed per instance, so a Summary metric cannot be aggregated across instances; Prometheus cannot merge pre-computed quantiles. This satisfies the stem's constraint by preventing cross-instance aggregation, unlike Histograms, which expose bucket counters that aggregate correctly server-side.

It is the deadline for the scrape response.

Identical label sets on both sides give PromQL a one-to-one match by default, pairing each left-hand series with the right-hand series sharing the same labels. No on() or ignoring() clause is needed, so arithmetic between matching vectors works directly.

Client libraries expose application metrics on a `/metrics` endpoint, letting Prometheus pull counter, gauge, histogram and summary values during each scrape. This satisfies the stem's instrumentation requirement: the library is the component embedded in your code that records and publishes measurements before the Prometheus server retrieves them.

Native histograms expose bucket counts through a sparse, structured sample format that histogram_quantile() parses natively, so quantile estimation works without the classic le-labelled bucket series. This satisfies the stem's requirement that the function evaluate native histograms directly.

The `repeat_interval` parameter sets how frequently Alertmanager re-notifies about an alert that remains firing after the initial notification, provided the alert has not been resolved or updated. It satisfies the scenario's requirement to define resend frequency for ongoing incidents, distinct from `group_interval`, which governs batching delays between different alert groups.

Remote write streams samples to an external system such as Thanos or Cortex, whose object storage retains data far beyond the local TSDB's retention window. This satisfies the long-term storage requirement, since Prometheus's own disk is sized for weeks, not years.

Errors counts failed requests, satisfying the Golden Signals requirement to track request failures. Unlike latency, which measures duration, errors quantify unsuccessful outcomes, whether explicit (HTTP 500s), implicit (malformed responses), or policy-based rejections. Google's SRE guidance names errors alongside latency, traffic, and saturation as the four signals, so this option belongs in the correct set.

Inhibition suppresses a lower-severity alert while a matching higher-severity alert is firing, using equal label matchers. This satisfies the stem's requirement by muting ServiceDown whenever GlobalNetworkOutage is active, preventing duplicate notification storms during broad outages.

Histogram buckets are cumulative: every bucket whose upper bound is greater than or equal to the observed value increments. With 0.7 seconds, the 1 and 5 buckets increment, while 0.1 and 0.5 do not, since 0.7 exceeds their bounds.

Using `on(job)` restricts matching to the `job` label, ignoring the differing `instance` label that would otherwise prevent a match. `group_left(instance)` then carries the left vector's `instance` label into the result, satisfying the requirement to retain it. This is the standard many-to-one matching configuration.

A Counter is a monotonically increasing value that only goes up, resetting to zero on restart, which suits a cumulative request tally. Tracking requests received requires this monotonic behaviour, and rate() or increase() can then derive per-second rates.

The 'on' modifier restricts matching to the label sets you explicitly name, so only series sharing those listed labels pair up. This lets vectors of differing cardinality align on a chosen subset of dimensions rather than every shared label.

Official client libraries expose metrics directly from application code in the Prometheus exposition format, so a scrape endpoint is served without a separate exporter. This satisfies the stem's requirement for a standard handling method, since instrumentation at source is the canonical approach for custom application metrics.

The up metric is synthesised by Prometheus for every scrape, reporting 1 when the target responded and 0 otherwise. It satisfies the stem's constraint because it is added automatically to all scraped targets, independent of exporter-supplied metrics.

Alertmanager evaluates group_by labels against each alert's label set. A label absent from an incoming alert resolves to an empty string, so all alerts missing that label collapse into a single group keyed by a null value.

The `!~` operator applies a negative regular-expression match, so `environment!~"staging"` excludes any series whose environment label matches the regex "staging", satisfying the stem's requirement for regex-based negation rather than the exact-match `!=` operator. This mirrors the positive `=~` operator but inverts the match result.

Using `start_http_server()` from the Prometheus Python client spins up a background HTTP endpoint that exposes metrics in the Prometheus text format, satisfying the requirement to expose custom exporter metrics without a separate web framework. It is the idiomatic, minimal-code approach for Python exporters.

A counter is a cumulative metric that only increases (or resets to zero on restart), and total HTTP requests processed fits this exactly. It monotonically accumulates, so rate() can compute requests per second over time.

Alerting rules contain PromQL expressions evaluated against the time series database; when the expression returns results for the specified duration, the alert fires. This satisfies the requirement that rules are defined using PromQL rather than a separate query language.

Inhibition rules suppress a target alert when a matching source alert is firing, using equal label matchers such as instance. This directly satisfies the stem's requirement to mute DiskSpaceLow while InstanceDown is active for the same host, which silences and routing cannot achieve.

'Too many active series' indicates cardinality growth, so identifying which label names contribute the most series pinpoints the offending metric before any dropping or relabelling. This diagnostic step precedes remediation such as label dropping or metric relabelling.

Exporters run alongside systems that natively expose no Prometheus metrics, translating their proprietary telemetry into the Prometheus text exposition format on an HTTP endpoint. Prometheus then scrapes that endpoint like any other target, bridging the format gap without altering the monitored application.

A Prometheus alerting rule's 'expr' field holds the PromQL expression; the alert fires when that expression's result satisfies the rule's 'for' duration. The threshold comparison itself is written inside the PromQL expression, not in a separate numeric field.

A short-lived batch job may exit before Prometheus's next scrape interval, so its metrics endpoint disappears and the samples are never collected. This directly satisfies the stem's constraint: the job finishes in seconds, far quicker than typical polling, requiring a push gateway or similar mechanism instead.

Alertmanager configuration routes alerts by matching label sets, so a route with a `service` matcher directs each alert to the correct receiver, satisfying the requirement to notify different teams per service label. This logic belongs in Alertmanager's routing tree, not in Prometheus scrape configs or Grafana dashboards.

Prometheus text exposition format lets the custom exporter present metrics that Prometheus can scrape natively, satisfying the interoperability requirement. Each metric needs HELP and TYPE annotations plus a consistent naming convention, so the collector parses values without transformation. This directly addresses the design consideration of choosing an output format compatible with the monitoring platform consuming the data.

Alertmanager is a distinct Prometheus architecture component: it receives firing alerts from the Prometheus server, then deduplicates, groups, silences and routes them to receivers such as email or PagerDuty. This matches the stem's request for a primary component.

node_exporter is the standard Prometheus exporter for Linux hosts, exposing CPU, memory, disk, filesystem and network metrics from /proc and /sys. It directly satisfies the requirement to track CPU usage alongside other hardware and OS metrics.

Targets expose metrics passively, so Prometheus discovers and scrapes them without any target-side configuration naming the server. This satisfies the stem's service-discovery constraint: targets remain unaware of Prometheus's location, unlike push models where each target must know the gateway endpoint.

A gauge represents a value that can rise or fall, making it the correct type for memory usage, which fluctuates as allocations and frees occur. Counters would be unsuitable because they only increase and cannot represent a current level.

The `expr` field holds the PromQL expression Prometheus evaluates on each evaluation interval; a non-empty result triggers the alert. It is mandatory in every alerting rule, satisfying the stem's requirement for a valid field alongside `alert` and `for`.

A range vector returns a set of time series, each holding multiple data points sampled over a specified time window, such as `[5m]`. This satisfies the stem's requirement for a valid PromQL data type, distinct from the instant vector's single sample per series.

An instant query evaluates at a single timestamp, so PromQL returns an instant vector holding one sample per matching series. This satisfies the stem's default behaviour, since a range vector only appears when the selector explicitly supplies a duration such as [5m].

Pushgateway lets short-lived batch jobs push their metrics to an intermediary that Prometheus then scrapes, since the job terminates before a normal pull scrape can occur. This satisfies the stem's constraint of a job finishing before scraping.

The --cluster.peer flag joins Alertmanager instances into a gossip cluster, so notification state and silences are shared; if communication between peers fails, each node still evaluates and dispatches its own alerts. This satisfies the requirement that alerts continue being sent during cluster partition.

Relabeling rewrites label sets during service discovery, so a drop action on high-variance labels removes them before ingestion. This directly lowers series cardinality, satisfying the stem's constraint of reducing the label combinations Prometheus must store and index.

Alertmanager's `inhibit_rules` block suppresses a target alert when a matching source alert is firing, using `equal` labels to scope the match. Setting `source_matchers: [alertname="InstanceDown"]`, `target_matchers: [alertname="HighErrorRate"]` and `equal: [instance]` silences HighErrorRate only for the same instance, satisfying the stem's per-instance constraint.

topk is a selection, not a fixed-size generator. When fewer than three series match the selector, it simply returns every matching series, ranked by value, with no synthetic padding or error, satisfying the scenario where the selector yields fewer results than k.

predict_linear() extrapolates a gauge's rate of change, so it requires a range vector of gauge samples plus a scalar number of seconds to forecast ahead. A single instant sample gives no slope, so the range vector is the essential input for projecting disk exhaustion.