Refer to the exhibit. If the error rate spikes to 2% for only 2 minutes, why does the alert not fire?
Exhibit
{
"displayName": "High Error Rate",
"conditions": [
{
"displayName": "Error rate > 1%",
"conditionThreshold": {
"aggregations": [
{
"alignmentPeriod": "60s",
"perSeriesAligner": "ALIGN_RATE"
}
],
"comparison": "COMPARISON_GT",
"filter": "metric.type=\"custom.googleapis.com/error_rate\" AND resource.type=\"k8s_container\"",
"duration": "300s",
"thresholdValue": 0.01
}
}
],
"alertStrategy": {
"notificationRateLimit": {
"period": "300s"
}
}
}Trap 1: The notification rate limit prevents the alert from firing.
Notification rate limit only limits repeated notifications after the alert fires.
Trap 2: The threshold value of 1% is too high.
Threshold is 1%, spike is 2%, so it exceeds.
Trap 3: The alignment period of 60s is too short.
Alignment period 60s is appropriate; duration is the issue.
- A
The notification rate limit prevents the alert from firing.
Why wrong: Notification rate limit only limits repeated notifications after the alert fires.
- B
The duration of 300s requires the condition to be met for 5 minutes.
The spike lasted only 2 minutes, insufficient to meet the duration requirement.
- C
The threshold value of 1% is too high.
Why wrong: Threshold is 1%, spike is 2%, so it exceeds.
- D
The alignment period of 60s is too short.
Why wrong: Alignment period 60s is appropriate; duration is the issue.