mirror of
https://github.com/grafana/grafana.git
synced 2025-09-25 16:03:46 +08:00
9941dfc2b6
* Update the Alerting HA docs * Update docs/sources/alerting/set-up/configure-high-availability/_index.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> * Update docs/sources/alerting/set-up/configure-high-availability/_index.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> * Update docs/sources/alerting/set-up/configure-high-availability/_index.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> * Update docs/sources/alerting/set-up/configure-high-availability/_index.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> * Update docs/sources/alerting/set-up/configure-high-availability/_index.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> * Update HA introduction paragraph * Apply prettier Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>
99 lines
2.8 KiB
Markdown
99 lines
2.8 KiB
Markdown
---
|
|
aliases:
|
|
- ../high-availability/enable-alerting-ha/
|
|
- ../unified-alerting/high-availability/
|
|
description: Enable alerting high availability
|
|
keywords:
|
|
- grafana
|
|
- alerting
|
|
- tutorials
|
|
- ha
|
|
- high availability
|
|
title: Enable alerting high availability
|
|
weight: 300
|
|
---
|
|
|
|
# Enable alerting high availability
|
|
|
|
You can enable alerting high availability support by updating the Grafana configuration file. If you run Grafana in a Kubernetes cluster, additional steps are required. Both options are described below.
|
|
|
|
## Enable alerting high availability in Grafana
|
|
|
|
### Before you begin
|
|
|
|
Since gossiping of notifications and silences uses both TCP and UDP port `9094`, ensure that each Grafana instance is able to accept incoming connections on these ports.
|
|
|
|
**To enable high availability support:**
|
|
|
|
1. In your custom configuration file ($WORKING_DIR/conf/custom.ini), go to the `[unified_alerting]` section.
|
|
2. Set `[ha_peers]` to the number of hosts for each Grafana instance in the cluster (using a format of host:port), for example, `ha_peers=10.0.0.5:9094,10.0.0.6:9094,10.0.0.7:9094`.
|
|
You must have at least one (1) Grafana instance added to the [`[ha_peer]` section.
|
|
3. Set `[ha_listen_address]` to the instance IP address using a format of `host:port` (or the [Pod's](https://kubernetes.io/docs/concepts/workloads/pods/) IP in the case of using Kubernetes).
|
|
By default, it is set to listen to all interfaces (`0.0.0.0`).
|
|
|
|
## Enable alerting high availability using Kubernetes
|
|
|
|
If you are using Kubernetes, you can expose the pod IP [through an environment variable](https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/) via the container definition.
|
|
|
|
```bash
|
|
env:
|
|
- name: POD_IP
|
|
valueFrom:
|
|
fieldRef:
|
|
fieldPath: status.podIP
|
|
```
|
|
|
|
1. Add the port 9094 to the Grafana deployment:
|
|
|
|
```yaml
|
|
ports:
|
|
- containerPort: 3000
|
|
name: http-grafana
|
|
protocol: TCP
|
|
- containerPort: 9094
|
|
name: grafana-alert
|
|
protocol: TCP
|
|
```
|
|
|
|
2. Add the environment variables to the Grafana deployment:
|
|
|
|
```yaml
|
|
env:
|
|
- name: POD_IP
|
|
valueFrom:
|
|
fieldRef:
|
|
fieldPath: status.podIP
|
|
```
|
|
|
|
3. Create a headless service that returns the pod IP instead of the service IP, which is what the `ha_peers` need:
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: grafana-alerting
|
|
namespace: grafana
|
|
labels:
|
|
app.kubernetes.io/name: grafana-alerting
|
|
app.kubernetes.io/part-of: grafana
|
|
spec:
|
|
type: ClusterIP
|
|
clusterIP: 'None'
|
|
ports:
|
|
- port: 9094
|
|
selector:
|
|
app: grafana
|
|
```
|
|
|
|
4. Make sure your grafana deployment has the label matching the selector, e.g. `app:grafana`.
|
|
|
|
5. Add in the grafana.ini:
|
|
|
|
```bash
|
|
[unified_alerting]
|
|
enabled = true
|
|
ha_listen_address = "${POD_IP}:9094"
|
|
ha_peers = "grafana-alerting.grafana:9094"
|
|
ha_advertise_address = "${POD_IP}:9094"
|
|
```
|