mirror of
https://github.com/grafana/grafana.git
synced 2025-09-18 12:32:51 +08:00
docs(alerting): stale alert instances updates (#103014)
* docs(alerting): stale alert instances updates * correct vale error
This commit is contained in:
@ -68,6 +68,24 @@ Alert instances will be routed for [notifications](ref:notifications) when they
|
||||
|
||||
{{< figure src="/media/docs/alerting/alert-rule-evaluation-overview-statediagram-v2.png" alt="A diagram of the alert instance states and when to route their notifications." max-width="750px" >}}
|
||||
|
||||
### Stale alert instances (MissingSeries)
|
||||
|
||||
The `No Data` state occurs when the alert rule query runs successfully but returns no data points at all.
|
||||
|
||||
An alert instance is considered stale if the query returns data but its dimension or series has disappeared for two evaluation intervals. In this case, the alert instance transitions to the **Normal (MissingSeries)** state as resolved, and is then evicted.
|
||||
|
||||
1. The alert rule runs and returns data for some label sets.
|
||||
|
||||
1. An alert instance that previously existed is now missing.
|
||||
|
||||
1. Grafana keeps the previous state of the alert instance for two evaluation intervals.
|
||||
|
||||
1. If it remains missing after two intervals, it transitions to the **Normal** state and sets **MissingSeries** in the `grafana_state_reason` annotation.
|
||||
|
||||
1. Stale alert instances in the **Alerting**, **No Data**, or **Error** states transition to the **Normal** state as **Resolved**, and are routed for notifications like other resolved alerts.
|
||||
|
||||
1. The alert instance is removed from the UI.
|
||||
|
||||
### `No Data` and `Error` alerts
|
||||
|
||||
When an alert rule evaluation results in a `No Data` or `Error` state, Grafana Alerting immediately creates a new alert instance —skipping the pending period—with the following additional labels:
|
||||
@ -82,12 +100,6 @@ You can manage these alerts like regular ones by using their labels to apply act
|
||||
|
||||
If the alert rule is configured to send notifications directly to a selected contact point (instead of using notification policies), the `DatasourceNoData` and `DatasourceError` alerts are also sent to that contact point. Any additional notification settings defined in the alert rule, such as muting or grouping, are preserved.
|
||||
|
||||
### Lifecycle of stale alert instances
|
||||
|
||||
An alert instance is considered stale if its dimension or series has disappeared from the query results entirely for two evaluation intervals.
|
||||
|
||||
Stale alert instances that are in the **Alerting**, **No Data**, or **Error** states transition to the **Normal** state as **Resolved**. Once transitioned, these resolved alert instances are routed for notifications like other resolved alerts.
|
||||
|
||||
## Modify the `No Data` or `Error` state
|
||||
|
||||
These states are supported only for Grafana-managed alert rules.
|
||||
@ -121,7 +133,7 @@ However, in situations where strict monitoring is critical, relying solely on th
|
||||
|
||||
Occasionally, an alert instance may be in a state that isn't immediately clear to everyone. For example:
|
||||
|
||||
- Stale alert instances in the `Alerting` state transition to the `Normal` state when the series disappear.
|
||||
- [Stale alert instances](#stale-alert-instances-missingseries) in the `Alerting` state transition to the `Normal` state when the series disappear.
|
||||
- If "no data" handling is configured to transition to a state other than `No Data`.
|
||||
- If "error" handling is configured to transition to a state other than `Error`.
|
||||
- If the alert rule is deleted, paused, or updated in some cases, the alert instance also transitions to the `Normal` state.
|
||||
@ -130,7 +142,7 @@ In these situations, the evaluation state may differ from the alert state, and i
|
||||
|
||||
The `grafana_state_reason` annotation is included in these situations, providing the reason that explains why the alert instance transitioned to its current state. For example:
|
||||
|
||||
- Stale alert instances in the `Normal` state include the `grafana_state_reason` annotation with the value **MissingSeries**.
|
||||
- [Stale alert instances](#stale-alert-instances-missingseries) in the `Normal` state include the `grafana_state_reason` annotation with the value **MissingSeries**.
|
||||
- If "no data" or "error" handling transitions to the `Normal` state, the `grafana_state_reason` annotation is included with the value **No Data** or **Error**, respectively.
|
||||
- If the alert rule is deleted or paused, the `grafana_state_reason` is set to **Paused** or **RuleDeleted**. For some updates, it is set to **Updated**.
|
||||
|
||||
|
Reference in New Issue
Block a user