Files
Pepe Cano ec42b2a361 Alerting docs: restructure Introduction (#84248)
* Rename `Data sources` title

* Relocate and rename `Introduction/Notification templates`

* Rename `alert-rules/alert-instances` to `alert-rules/multi-dimensional-alerts`

* Move `fundamentals/high-availability` to `setup/enable-ha`

* Fix 404 high-availability alerting link on Setup HA Grafana docs

* Move alert manager/contact poitns/notification templates within Notifications

* Remove `Alerting on numeric data`

* Restructure Introduction v2

* Continue Intro restructuring

* Update docs/sources/alerting/fundamentals/alert-rules/_index.md

Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>

* Complete contact point TODO

* Alias: alertManager

* Aliases `annotation-label` + content changes

* Aliases to `templating-labels-annotations`

* Aliases to `queries-conditions`

* Rename `rule-evaluation.md` file

* Aliases: `contact points`

* Aliases to `message-templating`

* Aliases to `alert-rules`

* Update links to new URL slugs

* Remove duplicated alias

* Remove trailing slash for external heading links

* Remove trailing slash in heading links to other grafana pages

* Change URL directory slug `fundamentals/notifications`

* rename title `Configure High Availability`

* Content changes

* Update docs/sources/alerting/fundamentals/alert-rules/_index.md

Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>

* Update docs/sources/alerting/set-up/configure-alert-state-history/index.md

Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>

* Update docs/sources/alerting/set-up/configure-high-availability/_index.md

Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>

* Update docs/sources/alerting/set-up/configure-alert-state-history/index.md

Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>

* Update docs/sources/alerting/set-up/configure-high-availability/_index.md

Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>

* Update docs/sources/alerting/set-up/configure-high-availability/_index.md

Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>

* Update docs/sources/alerting/set-up/configure-high-availability/_index.md

Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>

* Update docs/sources/alerting/fundamentals/alert-rules/_index.md

Co-authored-by: Jack Baldry <jack.baldry@grafana.com>

* Fix broken link reference

* Fix `queries-and-conditions`

* Fix `alert-rule-evaluation` ref link

* Fix aliases + inline doc comments

* Fix broken link

---------

Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>
Co-authored-by: Jack Baldry <jack.baldry@grafana.com>
2024-03-14 16:58:18 +01:00

3.4 KiB

canonical description keywords labels title weight
https://grafana.com/docs/grafana/latest/alerting/fundamentals/alert-rules/rule-evaluation/ Use alert rule evaluation to determine how frequently an alert rule should be evaluated and how quickly it should change its state
grafana
alerting
evaluation
products
cloud
enterprise
oss
Alert rule evaluation 108

Alert rule evaluation

Use alert rule evaluation to determine how frequently an alert rule should be evaluated and how quickly it should change its state.

To do this, you need to make sure that your alert rule is in the right evaluation group and set a pending period time that works best for your use case.

Evaluation group

Every alert rule is part of an evaluation group. Each evaluation group contains an evaluation interval that determines how frequently the alert rule is checked.

Data-source managed alert rules within the same group are evaluated one after the other, while alert rules in different groups can be evaluated simultaneously. This feature is especially useful when you want to ensure that recording rules are evaluated before any alert rules.

Grafana-managed alert rules are evaluated at the same time, regardless of alert rule group. The default evaluation interval is set at 10 seconds, which means that Grafana-managed alert rules are evaluated every 10 seconds to the closest 10-second window on the clock, for example, 10:00:00, 10:00:10, 10:00:20, and so on. You can also configure your own evaluation interval, if required.

Note:

Evaluation groups and alerts grouping in notification policies are two separate things. Grouping in notification policies allows multiple alerts sharing the same labels to be sent in the same time message.

Pending period

By setting a pending period, you can avoid unnecessary alerts for temporary problems.

In the pending period, you select the period in which an alert rule can be in breach of the condition until it fires.

Example

Imagine you have an alert rule evaluation interval set at every 30 seconds and the pending period to 90 seconds.

Evaluation will occur as follows:

[00:30] First evaluation - condition not met.

[01:00] Second evaluation - condition breached. Pending counter starts. Alert starts pending.

[01:30] Third evaluation - condition breached. Pending counter = 30s. Pending state.

[02:00] Fourth evaluation - condition breached. Pending counter = 60s Pending state.

[02:30] Fifth evaluation - condition breached. Pending counter = 90s. Alert starts firing

If the alert rule has a condition that needs to be in breach for a certain amount of time before it takes action, then its state changes as follows:

  • When the condition is first breached, the rule goes into a "pending" state.

  • The rule stays in the "pending" state until the condition has been broken for the required amount of time - pending period.

  • Once the required time has passed, the rule goes into a "firing" state.

  • If the condition is no longer broken during the pending period, the rule goes back to its normal state.

Note:

If you want to skip the pending state, you can simply set the pending period to 0. This effectively skips the pending period and your alert rule will start firing as soon as the condition is breached.

When an alert rule fires, alert instances are produced, which are then sent to the Alertmanager.