PCA Alerting And Dashboarding — 20 Questions

Question 1

Which PromQL function is used to convert a gauge into a rate of increase?

Accepted Answer

rate(). The rate() function is specifically designed to calculate the per-second rate of increase for counter metrics.

Answer

increase()

Answer

avg()

Answer

sum()

Question 2

What does the 'resolve_timeout' setting in Alertmanager do?

Accepted Answer

It determines how long to wait before sending a resolved notification. If an alert stops firing, Alertmanager waits for the resolve_timeout duration to ensure the alert has truly resolved before notifying about the resolution.

Answer

It sets the timeout for external API calls

Answer

It defines how long to store alert history

Answer

It suppresses duplicate alerts

Question 3

How can you view the current status of all alerts directly from the Prometheus server CLI?

Accepted Answer

By visiting the /alerts endpoint in the browser. The Prometheus UI on port 9090 has an 'Alerts' tab that shows every configured rule and its current status.

Answer

By tailing the prometheus.log file

Answer

Using 'curl' on the TSDB files

Answer

Using 'promtool check rules'

Question 4

Which of these is the correct syntax for a simple Alertmanager route?

Accepted Answer

route: { receiver: 'email' }. The route uses a 'match' or 'match_re' block to filter alerts and a 'receiver' field to specify where they go.

Answer

receiver: 'email'

Answer

alerts: { to: 'email' }

Answer

route: [ 'email' ]

Question 5

How can you debug an Alertmanager routing issue?

Accepted Answer

Use 'amtool alert show'. The Alertmanager web UI allows you to see which alerts are arriving and provides information on which route they matched.

Answer

Check the Prometheus metrics for total alerts

Answer

Restart the server

Answer

Delete all silences

Question 6

Which THREE of the following fields exist in the Alertmanager 'inhibit_rules' block?

Accepted Answer

source_match. Inhibition rules require 'source_match', 'target_match', and 'equal' fields to function.

Answer

match_re

Answer

repeat

Question 7

Which THREE of the following are standard ways to integrate Grafana with Prometheus?

Accepted Answer

Using PromQL to query metrics in panels. Grafana connects to Prometheus via the Data Source interface, supports PromQL queries in panels, and can visualize metrics from Prometheus exporters.

Answer

Installing the Prometheus plugin in Grafana

Answer

Configuring alert rules in the Prometheus YAML file

Question 8

You have a Prometheus alerting rule that triggers too frequently during flapping states. Which feature should you use to prevent this without silencing the alert entirely?

Accepted Answer

The 'for' field. The 'for' field in an alerting rule allows you to specify a duration before the alert transitions from 'pending' to 'firing', which filters out short-lived spikes.

Answer

The 'resolve_timeout' parameter

Answer

The 'group_wait' parameter

Answer

The 'inhibit_rules' block

Question 9

You are creating a Grafana dashboard and need to display a gauge showing the current CPU usage percentage from a Prometheus data source. Which function is most appropriate?

Accepted Answer

No function (raw query). For a current value, you do not need an aggregation over time; you just query the metric directly.

Answer

predict_linear()

Answer

sum_over_time()

Answer

rate()

Question 10

You have a recording rule named 'job:node_cpu:avg_rate_5m'. Where is this metric stored once the rule is executed?

Accepted Answer

In the Prometheus TSDB. Recording rules store the result of the expression as a new time series in the Prometheus TSDB.

Answer

In the Alertmanager memory

Answer

In the Grafana database

Answer

In a local flat file

Question 11

When configuring Alertmanager 'group_by' settings, what happens if you include a label that is not present on an incoming alert?

Accepted Answer

The alert is grouped under a null value. Alertmanager groups alerts based on the defined labels. If a label is missing, the alert is still processed, but it will fall into a 'null' or 'none' bucket for that specific grouping key.

Answer

The alert is dropped

Answer

The alert triggers an error in the logs

Answer

The routing rule fails

Question 12

You need to prevent an alert from firing if a maintenance window is active. How should you approach this in Alertmanager?

Accepted Answer

Use a Silence. Silences are the standard way to prevent specific alerts from firing during a known maintenance period.

Answer

Delete the alerting rule

Answer

Modify the routing tree

Answer

Add a 'skip' label to the rule

Question 13

You have a global Alertmanager configuration where you want to suppress 'DiskSpaceLow' alerts if 'InstanceDown' is active for the same host. Which mechanism accomplishes this?

Accepted Answer

Inhibition rules. Inhibition rules in Alertmanager are designed to silence alerts based on labels that match between the firing alert and the inhibited alert.

Answer

Route tree

Answer

Repeat intervals

Answer

Grouping

Question 14

You notice that recording rules are consuming too much CPU on your Prometheus server. What is the most effective way to reduce the load while keeping data available?

Accepted Answer

Increase the evaluation_interval. Recording rules are evaluated at the 'evaluation_interval'. Increasing this interval reduces CPU usage by running the queries less frequently.

Answer

Decrease the evaluation_interval

Answer

Move recording rules to Alertmanager

Answer

Delete all indices

Question 15

Which file format is used to define Prometheus alerting rules?

Accepted Answer

YAML. Prometheus configuration and rules files use the YAML format.

Answer

JSON

Answer

TOML

Answer

XML

Question 16

You are seeing 'Alerting rule evaluation error' in your logs. What is the most likely cause?

Accepted Answer

Invalid PromQL in the expression. Syntactic errors in the PromQL query within an alerting rule will cause evaluation failures.

Answer

Alertmanager is down

Answer

Too many dashboards

Answer

Prometheus is out of disk space

Question 17

Which Grafana panel type is best for showing a trend of a metric over time?

Accepted Answer

Time series. The Time series panel is the standard for plotting metrics over time.

Answer

Table

Answer

Stat

Answer

Bar gauge

Question 18

What is the purpose of the 'labels' field in an alerting rule?

Accepted Answer

To add metadata for routing and filtering. Labels allow you to attach metadata to the alert, which can then be used in Alertmanager for routing and grouping.

Answer

To define the alert message body

Answer

To select the target server

Answer

To set the alert severity level

Question 19

When using the 'repeat_interval' in Alertmanager, what does it define?

Accepted Answer

How often to resend firing alerts. The repeat_interval defines how long to wait before re-sending a notification for an alert that is still active.

Answer

The timeout for the receiver

Answer

The TTL of the alert

Answer

How long to wait before firing

Question 20

You want to notify different teams based on the 'service' label of an alert. Where should you configure this logic?

Accepted Answer

In the Alertmanager configuration. The Alertmanager configuration file contains the routing tree that maps alerts to receivers based on labels.

Answer

In the node exporter config

Answer

In the Grafana data source

Answer

In the Prometheus alerting rule