9b91be739d
* WIP - Restructure recording rule pages * Add more `Recording rules` use cases * Update `Intro > Alert rules`. Minor style changes and updates to the recording rules section * Minor verb change * Switch `aliases` and fix broken link * Bold for clarity * Complete `Create recording rules in Grafana Alerting` * Complete data source-managed recording rules * Final update
10 KiB
| aliases | canonical | description | keywords | labels | title | weight | refs | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
https://grafana.com/docs/grafana/latest/alerting/fundamentals/alert-rules/ | Learn about alert rules |
|
|
Alert rules | 100 |
|
Alert rules
An alert rule is a set of evaluation criteria for when an alert rule should fire. An alert rule consists of:
- Queries and expressions that select the data set to evaluate.
- A condition (the threshold) that the query must meet or exceed to trigger the alert instance.
- An interval that specifies the frequency of alert rule evaluation and a duration indicating how long the condition must be met to trigger the alert instance.
- Other customizable options, for example, setting what should happen in the absence of data, notification messages, and more.
Grafana supports two different alert rule types: Grafana-managed alert rules and data source-managed alert rules.
Grafana-managed alert rules
Grafana-managed alert rules are the most flexible alert rule type. They allow you to create alert rules that can act on data from any of the supported data sources, and use multiple data sources in a single alert rule.
{{< figure src="/media/docs/alerting/grafana-managed-alerting-architecture.png" max-width="750px" caption="How Grafana-managed alerting works by default" >}}
- Alert rules are created within Grafana and query one or more data sources.
- Alert rules are evaluated by the Alert Rule Evaluation Engine from within Grafana.
- Firing and resolved alert instances are forwarded to handle their notifications.
Supported data sources
Grafana-managed alert rules can query backend data sources if Grafana Alerting is enabled by specifying {"backend": true, "alerting": true} in the plugin.json file.
Find the public data sources supporting Alerting in the Grafana Plugins directory.
Data source-managed alert rules
Data source-managed alert rules can query Prometheus-based data sources, such as Grafana Mimir or Grafana Loki. Alert rules are stored within the data source when the Ruler API is enabled (e.g., Mimir Ruler API or Loki Ruler API).
In this setup, the distributed architecture can provide high-availability and fault tolerance.
{{< figure src="/media/docs/alerting/mimir-managed-alerting-architecture-v2.png" max-width="750px" caption="Mimir-managed alerting architecture" >}}
- Alert rules are created and stored within the data source itself.
- Alert rules can only query Prometheus-based data.
- Alert rules are evaluated by the Alert Rule Evaluation Engine.
- Firing and resolved alert instances are forwarded to handle their notifications.
Recording rules
A recording rule pre-compute frequently used or computationally expensive queries, and saves the results as a new time series metric.
The new metric can then be used in alert rules and dashboards to optimize their queries.
Similar to alert rules, recording rules are evaluated periodically. For more details, refer to Create recording rules.
Comparison between alert rule types
When choosing which alert rule type to use, consider the following comparison between Grafana-managed and data source-managed alert rules.
Feature |
Grafana-managed alert rule |
Data source-managed alert rule |
|---|---|---|
| Create alert rules |
Yes | No. Only query Prometheus-based data sources. |
| Mix and match data sources | Yes | No |
| Add expressions to transform |
Yes | No |
| Use images in alert notifications | Yes | No |
| Support for recording rules | Yes | Yes |
| Organization | Organize and manage access with folders | Use namespaces |
| Scaling | More resource intensive, depend on the database, and are likely to suffer from transient errors. They only scale vertically. | Store alert rules within the data source itself and allow for “infinite” scaling. Generate and send alert notifications from the location of your data. |
| Alert rule evaluation and delivery | Alert rule evaluation and delivery is done from within Grafana, using an external Alertmanager; or both. | Alert rule evaluation and alert delivery is distributed, meaning there is no single point of failure. |