Prometheus Fundamentals practice questions

TSDB block corruption

Why it fails: Block corruption usually leads to missing data in queries, not gaps related to interval changes.

Stale markers not being processed

Why it fails: Stale markers are generated by the server, not by changing the scrape interval.

Too many labels on the metric

Why it fails: High cardinality slows down performance but does not cause gaps based on interval changes.

scrape_configs

Why it fails: scrape_configs defines targets, jobs and scrape intervals, but renaming metrics requires metric_relabel_configs nested within a job. It is tempting because scrape_configs is where per-job scrape behaviour is configured, yet it cannot rewrite metric names during ingestion.

remote_write

Why it fails: remote_write configures forwarding of already-scraped samples to remote endpoints; renaming must occur before storage, during ingestion. It is tempting because remote_write manipulates series on the way out, but it cannot alter metric names at scrape time.

target_relabel_configs

Why it fails: target_relabel_configs rewrites target labels before scraping, not metric names on ingested samples. It is tempting because it sits in the same relabeling family, but metric renaming requires metric_relabel_configs, which acts on samples after the scrape.

It calculates the average change

Why it fails: rate() is not intended for gauges.

It works normally

Why it fails: rate() requires monotonically increasing counter samples; a fluctuating gauge's decreases are treated as counter resets, so results are spurious rather than normal. It works correctly only on counters, like http_requests_total. For gauges, delta() gives the change between first and last sample.

It calculates the derivative

Why it fails: rate() computes per-second average increase with counter-reset handling, not a mathematical derivative; on a fluctuating gauge it yields distorted values. deriv() is the function that actually calculates a gauge's per-second derivative via linear regression. rate() suits counters such as bytes_total.

region

Why it fails: Prometheus attaches instance, job and metric-name labels during scraping; region is not among them, so it must come from relabelling or target metadata. It is tempting because cloud service discovery does expose region as a meta-label, and would be correct if relabelling promoted __meta_* region values.

job

Why it fails: The job label is usually defined by the user in config.

host

Why it fails: Prometheus adds instance, not host, to identify the scrape endpoint; host is not an automatic label. It is tempting because host is the conventional term in many monitoring systems, and would be correct only if relabelling mapped the target address onto a host label.

Check the /metrics endpoint

Why it fails: The metrics endpoint displays prometheus internal stats.

View /status/config

Why it fails: The standard path is /config.

Run a PromQL query

Why it fails: PromQL cannot query the server's own config file.

Look at the PID file

Why it fails: The PID file only contains the process ID.

It keeps the first one received

Why it fails: This leads to unpredictable behavior.

It averages them

Why it fails: Prometheus does not perform automatic averaging.

It keeps both values

Why it fails: Prometheus does not store duplicate series.

5s

Why it fails: A 5s timeout is shorter than the 15s interval, so slow scrapes abort prematurely and targets appear down despite responding. Prometheus defaults scrape_timeout to the scrape_interval when unset. 5s would suit a much shorter interval, such as 10s or below.

30s

Why it fails: A 30s timeout exceeds the 15s scrape_interval, so a hung scrape delays the next cycle, causing missed scrapes and gaps. Prometheus caps scrape_timeout at the scrape_interval. 30s would be valid only where the interval is at least 30s.

15s

Why it fails: Setting scrape_timeout equal to the 15s scrape_interval leaves no margin; a scrape that overruns delays the following cycle, producing missed collections. Prometheus recommends a timeout below the interval, typically around 10s. Equality would be acceptable only if scrapes always finish instantly.

It writes to the WAL first

Why it fails: While WAL is for local durability, remote_write has its own buffering.

It disables local storage

Why it fails: remote_write runs alongside local storage rather than replacing it; disabling local storage is neither required nor how delivery is assured. It is tempting as a way to avoid duplicate storage, but Prometheus still writes to its local TSDB and WAL while forwarding samples.

It requires manual intervention on failure

Why it fails: Remote write retries failed sends from a write-ahead log with backoff, so delivery is automatic rather than manual. Manual intervention describes neither the queue nor the receiver; it would only apply to bespoke, non-Prometheus pipelines where an operator replays failed batches by hand.

/alerts

Why it fails: /alerts shows alerting status.

/status/targets

Why it fails: The path is simply /targets.

/graph

Why it fails: /graph is for executing queries.

Histograms are computed on the client

Why it fails: Summaries are computed on the client.

Summaries are computed on the server

Why it fails: Histograms use server-side aggregation.

Summaries cannot be aggregated

Why it fails: Summaries do support aggregation via quantiles, so the claim is factually wrong; the real axis is that Summary quantiles are computed client-side and cannot be aggregated across instances, unlike Histogram buckets. It is tempting because that client-side limitation is often misstated as a total inability to aggregate.

global.labels

Why it fails: Global labels are added to all scraped targets.

remote_write.labels

Why it fails: These labels are for remote write destinations.

relabel_configs

Why it fails: relabel_configs are for dynamic transformation, not static additions.

max_series

Why it fails: No such parameter exists; the head block's series count is bounded by max_series_per_metric-style limits or sample limits, not max_series. It tempts because the name mirrors the intent, but the actual head-block guard is enforced through the TSDB's per-metric series cap and out-of-order sample handling.

series_limit

Why it fails: series_limit is not a Prometheus configuration parameter; the head block is bounded by max_series_per_metric and related TSDB flags. The name suggests a direct cap, which is why it tempts, but the real limit is applied per metric rather than as a global series ceiling.

max_cardinality

Why it fails: max_cardinality is not a Prometheus head-block parameter; cardinality limits are set via max_series_per_metric or sample-based guards. It tempts because cardinality is the concept being bounded, but the actual mechanism caps series per metric name rather than total head cardinality.

1m

Why it fails: Not the standard default.

1s

Why it fails: Too aggressive.

24h

Why it fails: Too infrequent.

scrape interval

Why it fails: Scrape interval is defined per job in scrape_configs, not in relabel_configs, which only rewrite labels and control target selection. Scrape interval is the correct setting when tuning collection frequency, but it cannot be altered through relabeling rules.

retention period

Why it fails: Retention period is configured via command-line flags such as --storage.tsdb.retention.time, not relabel_configs, which rewrites target labels during service discovery. It is tempting because retention governs how long samples persist, so it feels like scrape configuration, but it would be the right choice only when tuning TSDB storage duration.

alerting logic

Why it fails: Alerting logic lives in Prometheus alerting rules evaluated by the rule manager, not in relabel_configs, which only mutate target labels before scraping. It is tempting because relabelling can drop targets and thus affect which alerts fire, but it would be correct only for controlling label-based target selection during discovery.

Number of dashboards

Why it fails: Dashboards live in the browser.

Scrape interval

Why it fails: Scrape interval changes sample frequency, not the number of active series held in memory, so it does not directly drive Prometheus memory usage. It tempts because shorter intervals increase ingestion load and disk writes, but memory is governed by active time series, label cardinality and retention, which the stem asks about.

percentile

Why it fails: Percentile is a Summary label, not a Histogram label; Histograms expose le (bucket boundaries) and _bucket, _sum and _count series. It is tempting because both metric types track distributions, and percentile appears in PromQL queries against histogram data, but it is not a label attached to Histogram metrics.

quantile

Why it fails: Associated with Summaries.

range

Why it fails: Not a label.

receivers

Why it fails: The route block defines the routing tree; receivers only declare named notification integrations such as Slack. It tempts because Slack configuration lives inside a receiver, but matching, grouping and child routes are expressed under route, not receivers.

inhibit_rules

Why it fails: inhibit_rules suppresses notifications for one alert while a related alert fires; it never builds the routing tree. The route block defines that tree, matching labels to receivers such as Slack. Inhibition is tempting when you want to silence downstream alerts during an outage, which is its actual purpose.

global

Why it fails: global sets defaults inherited across the configuration, such as smtp_smarthost or resolve_timeout, not the routing tree. The route block defines that tree and its child routes. Global is tempting because it configures shared settings, and it would be correct for defaults applied to every receiver.

XML

Why it fails: Prometheus parses its main configuration as YAML, so an XML file is rejected at startup. XML is tempting because it is a widespread structured configuration format used by other monitoring and enterprise tooling, and would be the right choice when integrating with systems whose configuration schemas are defined in XML.

JSON

Why it fails: Prometheus loads its main configuration from YAML, not JSON, so a JSON file fails to parse. JSON is tempting because it is a common machine-readable configuration and data-exchange format, and would be correct for API payloads or service-discovery response bodies rather than the prometheus.yml configuration itself.

TOML

Why it fails: Prometheus expects YAML syntax in its main configuration file, so TOML keys are not recognised and startup fails. TOML is tempting because it is a legitimate, readable configuration format used by tools such as the Prometheus Alertmanager's predecessor ecosystems and many Go utilities, making it a plausible-looking alternative.

The metric name

Why it fails: A metric name is the first part of a time series identifier, not a sample; a sample is a timestamped value. It is tempting because every sample carries a metric name, and naming metrics is central to PromQL queries, but the name alone holds no value or timestamp.

A label key-value pair

Why it fails: A label key-value pair forms part of a series' identity, not a sample; a sample is a float value paired with a millisecond timestamp. It is tempting because labels distinguish series and appear in every query result, but they describe the series rather than constitute the recorded measurement itself.

An alert rule

Why it fails: An alert rule is an evaluation configuration that fires notifications; it is not a data point. It is tempting because alerting consumes samples and is configured alongside them, but a sample is strictly a timestamped value appended to a time series, independent of any rule.

dns_sd_configs

Why it fails: dns_sd_configs resolves DNS SRV or A records into targets; it cannot query the Kubernetes API for pod labels. It tempts because service discovery is the right concept, and it would be correct for discovering targets published through DNS records outside Kubernetes.

static_configs

Why it fails: static_configs lists fixed host:port targets and performs no label-based discovery, so pods labelled app=frontend would never be found. It tempts because it is the simplest discovery mechanism, and it would be correct for a small, unchanging set of known endpoints.

file_sd_configs

Why it fails: Kubernetes pod discovery requires kubernetes_sd_configs with a role of pod; file_sd_configs reads target lists from files on disk, so it cannot query the API server for label selectors. It is tempting because file-based discovery suits static or externally generated target lists, such as VMs or batch jobs.