mirror of
https://github.com/grafana/grafana.git
synced 2025-07-28 19:42:44 +08:00
![renovate[bot]](/assets/img/avatar_default.png)
* Update dependency prettier to v3.6.2 * run prettier --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Ashley Harrison <ashley.harrison@grafana.com>
168 lines
9.5 KiB
Markdown
168 lines
9.5 KiB
Markdown
---
|
||
aliases:
|
||
- ../../../configure-notifications/manage-contact-points/integrations/configure-oncall/ # /docs/grafana/<GRAFANA_VERSION>/alerting/configure-notifications/manage-contact-points/integrations/configure-oncall/
|
||
- ../../../alerting-rules/manage-contact-points/configure-oncall/ # /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/manage-contact-points/configure-oncall/
|
||
- ../../../alerting-rules/manage-contact-points/integrations/configure-oncall/ # /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/manage-contact-points/integrations/configure-oncall/
|
||
- ../configure-oncall/ # /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/manage-contact-points/configure-oncall/
|
||
canonical: https://grafana.com/docs/grafana/latest/alerting/configure-notifications/manage-contact-points/integrations/configure-irm/
|
||
description: Configure the Alerting - Grafana IRM integration to connect alerts generated by Grafana Alerting with Grafana IRM
|
||
keywords:
|
||
- grafana
|
||
- alerting
|
||
- oncall
|
||
- irm
|
||
- integration
|
||
labels:
|
||
products:
|
||
- oss
|
||
- enterprise
|
||
menuTitle: Grafana IRM
|
||
title: Configure Grafana IRM for Alerting
|
||
weight: 120
|
||
refs:
|
||
configure-grafana-alerts:
|
||
- pattern: /docs/grafana/
|
||
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/alerting-rules/create-grafana-managed-rule/
|
||
- pattern: /docs/grafana-cloud/
|
||
destination: /docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules/create-grafana-managed-rule/
|
||
pending-period:
|
||
- pattern: /docs/grafana/
|
||
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/fundamentals/alert-rule-evaluation/#pending-period
|
||
- pattern: /docs/grafana-cloud/
|
||
destination: /docs/grafana-cloud/alerting-and-irm/alerting/fundamentals/alert-rule-evaluation/#pending-period
|
||
timing-options:
|
||
- pattern: /docs/grafana/
|
||
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/fundamentals/notifications/group-alert-notifications/#timing-options
|
||
- pattern: /docs/grafana-cloud/
|
||
destination: /docs/grafana-cloud/alerting-and-irm/alerting/fundamentals/notifications/group-alert-notifications/#timing-options
|
||
configure-contact-points:
|
||
- pattern: /docs/grafana/
|
||
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/configure-notifications/manage-contact-points/
|
||
- pattern: /docs/grafana-cloud/
|
||
destination: /docs/grafana-cloud/alerting-and-irm/alerting/configure-notifications/manage-contact-points/
|
||
irm:
|
||
- pattern: /docs/
|
||
destination: /docs/grafana-cloud/alerting-and-irm/irm/
|
||
irm-alert-templates:
|
||
- pattern: /docs/
|
||
destination: /docs/grafana-cloud/alerting-and-irm/irm/configure/escalation-routing/alert-templates/
|
||
irm-escalation-chains:
|
||
- pattern: /docs/
|
||
destination: /docs/grafana-cloud/alerting-and-irm/irm/configure/escalation-routing/escalation-chains/
|
||
irm-configure-integrations:
|
||
- pattern: /docs/
|
||
destination: /docs/grafana-cloud/alerting-and-irm/irm/configure/integrations/configure-integrations/
|
||
webhook-contact-point:
|
||
- pattern: /docs/grafana/
|
||
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/configure-notifications/manage-contact-points/integrations/webhook-notifier
|
||
- pattern: /docs/grafana-cloud/
|
||
destination: /docs/grafana-cloud/alerting-and-irm/alerting/configure-notifications/manage-contact-points/integrations/webhook-notifier
|
||
webhook-json-payload:
|
||
- pattern: /docs/grafana/
|
||
destination: /docs/grafana/<GRAFANA_VERSION>/alerting/configure-notifications/manage-contact-points/integrations/webhook-notifier/#json-payload
|
||
- pattern: /docs/grafana-cloud/
|
||
destination: /docs/grafana-cloud/alerting-and-irm/alerting/configure-notifications/manage-contact-points/integrations/webhook-notifier/#json-payload
|
||
irm-contact-point-in-gc:
|
||
- pattern: /docs/
|
||
destination: /docs/grafana-cloud/alerting-and-irm/alerting/configure-notifications/manage-contact-points/integrations/configure-irm
|
||
---
|
||
|
||
[//]: <> (The IRM instructions are different for Grafana Cloud, so this page is currently skipped from Cloud docs.)
|
||
|
||
# Configure Grafana IRM for Alerting
|
||
|
||
In Grafana OSS and Grafana Enterprise, you can use a webhook contact point to send alerts to [Grafana IRM](ref:irm). Grafana IRM can then route alerts based on escalation chains for your team's workflows.
|
||
|
||
The alert notification flow is as follows:
|
||
|
||
<sup>\*</sup> **Grafana OSS/Enterprise** (send webhook alerts)->**Grafana Cloud IRM** (route via [escalation chains](ref:irm-escalation-chains))
|
||
|
||
{{< admonition type="note" >}}
|
||
|
||
The Free Forever plan in Grafana Cloud IRM includes 3 IRM active users per month.
|
||
|
||
These instructions apply only to Grafana OSS and Grafana Enterprise. To configure IRM for Grafana Cloud Alerting, refer to the [Grafana Cloud documentation](ref:irm-contact-point-in-gc).
|
||
|
||
{{< /admonition >}}
|
||
|
||
## Configure an integration in Grafana IRM
|
||
|
||
First, enable an integration in IRM to accept alerts from Grafana Alerting. You can either create a new integration or use an existing **Alertmanager** or **Webhook** integration in IRM.
|
||
|
||
To create the integration, follow the same steps as described in [Configure an OnCall integration in IRM](ref:irm-configure-integrations):
|
||
|
||
1. Navigate to **Alerts & IRM** -> **IRM** -> **Integrations**.
|
||
1. Click **+ New integration**.
|
||
1. Select either **Alertmanager** or **Webhook** integration from the list.
|
||
- **Alertmanager** integration – Includes preconfigured IRM templates for processing Grafana and Prometheus alerts.
|
||
|
||
- **Webhook** integration – Uses default IRM templates for general alert processing.
|
||
|
||
1. Provide a title, description, and assign it to a team, then click **Create Integration**.
|
||
1. On the integration details page, copy the **HTTP Endpoint** URL to use in the next section.
|
||
|
||
## Configure a webhook contact point in Grafana Alerting
|
||
|
||
Next, configure the contact point in Grafana Alerting to send alert notifications to Grafana IRM.
|
||
|
||
In **Grafana OSS** and **Grafana Enterprise**, you need to create a **Webhook** contact point using the HTTP endpoint URL you copied in the previous step.
|
||
|
||
1. Navigate to **Alerts & IRM** -> **Alerting** -> **Contact points**.
|
||
1. Click **+ Add contact point**.
|
||
1. Enter a name for the contact point.
|
||
1. From the **Integration** list, select **Webhook**.
|
||
1. In the **URL** field, enter the HTTP endpoint URL of the IRM integration.
|
||
1. Click **Save contact point**.
|
||
|
||
After configuring the contact point in Grafana Alerting and the integration in Grafana IRM, you can start testing the Alerting-to-IRM integration.
|
||
|
||
{{< figure src="/media/docs/alerting/test-alert-group-on-irm.png" max-width="750px" caption="Example of a test alert group in Grafana IRM" >}}
|
||
|
||
For more information, see:
|
||
|
||
- **[Configure contact points](ref:configure-contact-points)** – Learn how to test the integration and enable notifications in Alerting.
|
||
- **[Webhook contact point](ref:webhook-contact-point)** – Learn the format of the webhook payload and additional settings in Alerting.
|
||
- **[Configure IRM alert templates](ref:irm-alert-templates)** – Learn how to process the incoming webhook messages in IRM.
|
||
|
||
## Enable heartbeat monitoring in Grafana IRM (optional)
|
||
|
||
Enabling the heartbeat in the IRM integration acts as a monitoring for your Grafana Alerting setup. If Grafana Alerting stops sending alerts—due to downtime or misconfiguration—Grafana IRM creates a new alert group to notify you.
|
||
|
||
To set up heartbeat monitoring, you must enable the heartbeat in IRM and create an alert rule that continuously sends alerts to the heartbeat endpoint.
|
||
|
||
#### Enable the heartbeat for the IRM integration
|
||
|
||
1. In **IRM**, select the IRM integration you configured earlier.
|
||
1. Click the ⋮ (three dots) on top right, then select **Heartbeat Settings**.
|
||
1. Copy the **Endpoint** URL—this will be used in a new contact point in Grafana Alerting.
|
||
1. Set the **heartbeat interval**, time period after which Grafana IRM starts a new alert group if it doesn’t receive a heartbeat.
|
||
|
||
#### Create another Webhook contact point in Grafana Alerting
|
||
|
||
Follow the same steps as before to create a webhook contact point in Grafana Alerting:
|
||
|
||
1. Navigate to **Alerts & IRM** -> **Alerting** -> **Contact points**.
|
||
1. Click **+ Add contact point**.
|
||
1. Enter a name for the contact point.
|
||
1. From the **Integration** list, select **Webhook**.
|
||
1. In the **URL** field, enter the endpoint URL of the heartbeat.
|
||
1. Click **Save contact point**.
|
||
|
||
You can now click the **Test** button to send an alert to the heartbeat endpoint. In **IRM**, verify the heartbeat status in the **Hearbeat** column on the **Integrations** page.
|
||
|
||
{{< figure src="/media/docs/alerting/view-heartbeat-status-on-irm.png" max-width="750px" caption="Heartbeat status column in the Grafana IRM Integrations page" >}}
|
||
|
||
#### Create an alert rule in Grafana Alerting
|
||
|
||
Create a [Grafana-managed alert rule](ref:configure-grafana-alerts) with the following settings:
|
||
|
||
- **Always firing** – Use a query and alert condition that constantly fire. For example, select a Prometheus data source and set the query to `vector(1) > 0`.
|
||
- Configure a [pending period](ref:pending-period) that is shorter than the **hearbeat interval**.
|
||
- Choose the **webhook contact point** you created for the heartbeat to forward alerts.
|
||
- Adjust [timing options](ref:timing-options) in the alert rule or notification policy to ensure alerts are forwarded before the **heartbeat interval** elapses:
|
||
- **Group wait**: `0s`
|
||
- **Group interval**: `1s`
|
||
- **Repeat interval**: shorter than the **hearbeat interval**.
|
||
|
||
After it's created, the alert rule acts as a heartbeat, verifying that Grafana Alerting is running and sending alerts to Grafana IRM.
|