Alert rule
A saved condition that is checked repeatedly and triggers a notification or incident action when it stays true.
See it
What it is
An alert rule is a condition that a monitoring system evaluates on a schedule. It combines a query, a comparison, and usually a duration: '5xx rate above 2 percent for 5 minutes.' When that stays true, the rule fires and hands an event to something that can notify Slack, page the on-call, or start an automated incident action. The threshold is just the boundary; the rule is the whole executable instruction.
Reach for one when a measurable symptom needs a timely human response. Prometheus evaluates alerting rules, then Alertmanager groups, silences, and routes the results. Good rules name what users are experiencing, carry a severity and runbook link, and wait long enough to ignore a single noisy sample.
Gotcha: a rule can be technically correct and operationally useless. Missing data may mean zero, a broken exporter, or a dead service, and each needs different handling. A rule that fires without an action trains people to ignore it, while a rule with no recovery condition can flap between firing and resolved all night.
Ask AI for it
Create Prometheus alerting rules for this service. Write PromQL expressions for sustained 5xx rate, p95 latency, and missing heartbeat, give each rule a for duration, add severity and service labels, and add summary, description, and runbook_url annotations. Route warning alerts to Slack and critical alerts to PagerDuty through Alertmanager. Group duplicate alerts by service and alertname, include a recovery path, and add promtool tests covering firing, pending, resolved, and no-data cases.