48 questions · Alerting And Dashboarding · All types, answers revealed
Which TWO of the following are benefits of using recording rules?
Complex expressions become a single metric name.
Why this answer
Recording rules improve performance for dashboards and provide pre-computed metrics for alerting.
A team wants to visualize metrics from Prometheus in Grafana. What is the standard way to connect these two services?
Adding a data source is the native way Grafana queries Prometheus.
Why this answer
Grafana allows users to add Prometheus as a 'Data Source' by providing the URL of the Prometheus server.
You notice that recording rules are consuming too much CPU on your Prometheus server. What is the most effective way to reduce the load while keeping data available?
Increasing the interval decreases the frequency of calculations, reducing CPU overhead.
Why this answer
Recording rules are evaluated at the 'evaluation_interval'. Increasing this interval reduces CPU usage by running the queries less frequently.
What happens to your alerts if the Alertmanager configuration file contains a syntax error?
Alertmanager maintains operational stability by not loading broken configs.
Why this answer
If the configuration is invalid, Alertmanager will fail to reload, and it will continue using the last known good configuration.
Which Grafana feature allows you to see data from multiple Prometheus servers on a single panel?
Variables enable users to dynamically switch or select data sources in a panel.
Why this answer
Grafana allows you to define multiple data sources and even mix them within a dashboard, or use a data source proxy.
When configuring a dashboard panel, what is the significance of the 'Legend' field?
The legend field transforms raw metric labels into readable strings.
Why this answer
The legend field allows you to customize the labels displayed in the graph tooltip and legend section.
Which Grafana panel type is best for showing a trend of a metric over time?
Time series panels visualize how metrics change over time.
Why this answer
The Time series panel is the standard for plotting metrics over time.
When using the 'repeat_interval' in Alertmanager, what does it define?
It controls the frequency of notification reminders.
Why this answer
The repeat_interval defines how long to wait before re-sending a notification for an alert that is still active.
Which Alertmanager feature would you use to prevent a 'ServiceDown' alert from firing if a 'GlobalNetworkOutage' alert is already active?
Inhibition suppresses alerts based on the existence of another.
Why this answer
Inhibition rules allow one alert to suppress another based on matching labels.
When configuring Alertmanager 'group_by' settings, what happens if you include a label that is not present on an incoming alert?
Alerts with missing grouping labels are grouped together in a default bucket.
Why this answer
Alertmanager groups alerts based on the defined labels. If a label is missing, the alert is still processed, but it will fall into a 'null' or 'none' bucket for that specific grouping key.
You have a global Alertmanager configuration where you want to suppress 'DiskSpaceLow' alerts if 'InstanceDown' is active for the same host. Which mechanism accomplishes this?
Inhibit_rules match labels between a source alert and a target alert to suppress the latter.
Why this answer
Inhibition rules in Alertmanager are designed to silence alerts based on labels that match between the firing alert and the inhibited alert.
Where do you define the threshold for a Prometheus alerting rule?
The query expression contains the threshold logic.
Why this answer
The threshold is part of the PromQL query expression within the 'expr' field of the alerting rule.
You want to notify different teams based on the 'service' label of an alert. Where should you configure this logic?
Routing logic is defined in the Alertmanager config file.
Why this answer
The Alertmanager configuration file contains the routing tree that maps alerts to receivers based on labels.
Which TWO of the following are valid fields in a Prometheus alerting rule definition?
Mandatory query expression.
Why this answer
A rule must have an 'alert' name and an 'expr' query.
You want to ensure that if the Alertmanager cluster loses communication, alerts are still sent. Which configuration helps achieve this?
Clustering ensures Alertmanager instances share state and suppress duplicate notifications.
Why this answer
High Availability (HA) for Alertmanager is achieved by running multiple instances and using the '--cluster.peer' flag to link them.
You have two alerts: 'InstanceDown' and 'HighErrorRate'. You want to inhibit 'HighErrorRate' if 'InstanceDown' is firing for the same instance. Where do you configure this logic?
The Alertmanager config manages alert grouping, inhibition, and routing.
Why this answer
Inhibition rules are defined within the 'inhibit_rules' section of the Alertmanager configuration file.
Which file format is used to define Prometheus alerting rules?
Prometheus rules are defined in YAML files.
Why this answer
Prometheus configuration and rules files use the YAML format.
What is the purpose of a 'grouping' configuration in the Alertmanager 'route' block?
Grouping combines alerts sharing specified labels into one notification bundle.
Why this answer
Grouping categorizes multiple alerts into a single notification based on common label sets, reducing alert fatigue.
Which THREE of the following are valid Alertmanager routing tree parameters?
Define notification recipient.
Why this answer
Routes can filter by 'match', 'match_re', and define a 'receiver'.
You have a recording rule that uses 'sum() by (instance)'. Why might this be inefficient for large clusters?
Each unique instance label combination creates a new time series.
Why this answer
High cardinality labels, if not handled correctly, can lead to a massive number of time series, consuming excessive memory and storage.
How do you apply a global notification delay in Alertmanager?
group_wait provides the initial delay for grouping alerts.
Why this answer
The 'group_wait' parameter at the root route level defines the time to wait before sending an initial notification for a new alert group.
You are creating a Grafana dashboard and need to display a gauge showing the current CPU usage percentage from a Prometheus data source. Which function is most appropriate?
Queries like 'node_cpu_seconds_total' provide the instantaneous current value suitable for a gauge.
Why this answer
For a current value, you do not need an aggregation over time; you just query the metric directly.
You need to send critical alerts to a specific Slack channel while warning alerts go to email. Where should this routing logic be defined?
The route tree evaluates alerts against matchers to assign them to the correct receiver.
Why this answer
The Alertmanager 'routes' configuration tree uses matchers to determine which receiver handles specific alerts based on labels.
Which of the following best describes the function of the 'continue: true' setting within an Alertmanager route?
By default, routing stops at the first match; 'continue: true' enables further matching.
Why this answer
Setting 'continue: true' allows an alert to match multiple routes, ensuring it can be sent to multiple destinations if desired.
In Alertmanager, what is the role of the 'continue' field in a route?
It allows an alert to match multiple routes, enabling multi-destination routing.
Why this answer
If 'continue' is set to true, Alertmanager will match the alert against subsequent sibling routes instead of stopping at the first match.
What does the 'firing' state mean for an alert?
The condition met the threshold and the duration requirement.
Why this answer
An alert enters the 'firing' state when its condition expression evaluates to true for the specified 'for' duration.
Which TWO of the following are valid components of an Alertmanager configuration file?
Receivers define where alerts are sent.
Why this answer
A standard Alertmanager config file requires 'route' (the routing tree) and 'receivers' (the notification targets).
You have a Prometheus alerting rule that triggers too frequently during flapping states. Which feature should you use to prevent this without silencing the alert entirely?
Setting a 'for' duration ensures the condition must persist before the alert fires.
Why this answer
The 'for' field in an alerting rule allows you to specify a duration before the alert transitions from 'pending' to 'firing', which filters out short-lived spikes.
You have a recording rule that fails to evaluate because of a 'labels conflict'. What is the most likely cause?
Prometheus requires unique labels for each time series; conflicts occur if the result isn't unique.
Why this answer
A label conflict occurs when the recording rule tries to create a metric with labels that are already present or restricted, or when the aggregation produces duplicate label sets.
When setting up Alertmanager, what happens if you have no route defined for an alert?
The root route acts as the default catch-all.
Why this answer
If an alert matches no specific route, it will fall back to the root route of the configuration tree.
Which THREE of the following are valid Alertmanager grouping parameters?
Defines the labels to group by.
Why this answer
Alertmanager grouping is configured via 'group_by', 'group_wait', and 'group_interval'.
Which TWO of the following are necessary to successfully inhibit an alert in Alertmanager?
Defines the alert to be suppressed.
Why this answer
Inhibition requires a 'target_matchers' (the alert to be silenced) and 'source_matchers' (the alert that triggers the silence), plus common labels to correlate the two.
When integrating Grafana with Prometheus, what is the standard authentication method if Prometheus is behind a reverse proxy?
Basic Auth is the built-in, recommended standard for securing the data source connection.
Why this answer
Grafana supports various authentication methods; using a Basic Auth header or a proxy header is the standard way to securely connect to a protected Prometheus instance.
Which THREE of the following are standard ways to send notifications from Alertmanager?
Standard programmatic integration.
Why this answer
Webhook, Email, and PagerDuty are built-in, widely used receiver types.
Which of these is NOT a valid Alertmanager receiver type?
SQL is not a native alerting receiver.
Why this answer
While Alertmanager supports many integrations (Webhook, Email, PagerDuty), 'SQL' is not a native built-in receiver type.
What is the purpose of the 'labels' field in an alerting rule?
Labels are the primary mechanism for routing and grouping in Alertmanager.
Why this answer
Labels allow you to attach metadata to the alert, which can then be used in Alertmanager for routing and grouping.
Which THREE of the following are valid components within a Prometheus Alerting Rule file?
The PromQL expression used to evaluate the condition.
Why this answer
Prometheus alerting rules contain a 'groups' array, which contains individual 'rules' (alerts or recordings), and each alert rule must have an 'alert' name and 'expr'.
Which TWO of the following are valid ways to prevent alert flapping?
Extends the firing state to buffer against noise.
Why this answer
The 'for' duration and 'keep_firing_for' are both mechanisms to prevent alerts from toggling state too quickly.
Which tool allows you to visualize Prometheus alerts directly in a web UI?
The Alertmanager UI specifically lists firing and inhibited alerts.
Why this answer
The Prometheus built-in web UI provides an 'Alerts' tab to view current alert statuses.
You have a recording rule named 'job:node_cpu:avg_rate_5m'. Where is this metric stored once the rule is executed?
Recording rules result in new metrics being stored in the TSDB.
Why this answer
Recording rules store the result of the expression as a new time series in the Prometheus TSDB.
You are seeing 'Alerting rule evaluation error' in your logs. What is the most likely cause?
Syntax errors in the query expression prevent the rule from evaluating.
Why this answer
Syntactic errors in the PromQL query within an alerting rule will cause evaluation failures.
Why would you choose to create a recording rule for a complex PromQL query?
Recording rules improve performance for expensive queries.
Why this answer
Recording rules pre-calculate complex queries, making dashboards load faster by querying the pre-computed series instead of re-calculating the entire expression.
You are creating a recording rule to calculate the rate of requests over 5 minutes. Why would you prefer a recording rule over a direct dashboard query?
Precomputing expensive queries improves dashboard performance significantly.
Why this answer
Recording rules precompute expensive queries, reducing the load on Prometheus and speeding up dashboard loading times.
What is the purpose of the 'keep_firing_for' field in an alerting rule?
It extends the firing state to smooth out minor fluctuations.
Why this answer
It keeps an alert in the firing state for a specified duration after the underlying expression is no longer true, which helps prevent flapping.
Which annotation is commonly used to provide a human-readable description in an alert?
The 'description' annotation is used to provide details about the alert.
Why this answer
The 'summary' or 'description' annotations are standard practices for providing context in alert notifications.
You need to ensure that an alert remains 'firing' for 5 minutes before the Alertmanager is notified. Which field in the Prometheus alerting rule should you configure?
The 'for' field allows a duration to be specified before an alert is considered firing.
Why this answer
The 'for' field in a Prometheus alerting rule specifies the duration for which a condition must be true before the alert transitions from 'pending' to 'firing'.
You need to prevent an alert from firing if a maintenance window is active. How should you approach this in Alertmanager?
Silences allow you to mute specific alerts based on label matchers for a set time.
Why this answer
Silences are the standard way to prevent specific alerts from firing during a known maintenance period.
You have a recording rule that references a metric that doesn't exist. What is the impact?
If the expression finds no series, it simply does not create a new metric.
Why this answer
The recording rule will be evaluated, but because the expression returns no data, no new time series will be created for that rule.
Ready to test yourself?
Try a timed practice session using only Alerting And Dashboarding questions.