What is this feature?
Implements the POST endpoint for deleting imported Mimir Alertmanager configurations:
POST /api/convert/api/v1/alerts
The API endpoint creates the extra Alertmanager configuration with the provided identifier and matchers.
* declare dingding url as secret
patch raw settings before parsing because DingDing's config parser does not know about secrets
* fix integration test
---------
Co-authored-by: Yuri Tseretyan <yuriy.tseretyan@grafana.com>
What is this feature?
Fixes a bug when group-level query_offset and labels parameters are ignored and not saved
Why do we need this feature?
In the import API Prometheus YAML rule definitions are supported:
groups:
- name: group-1
interval: 1m
query_offset: 10m
labels:
severity: "warning"
rules:
- alert: Alert 0 > 0
expr: vector(0) > 0
But applying group-level labels and query_offset is broken and they are not saved right now because during the conversion of the API model to PrometheusRuleGroup they aren't saved to the new structure.
Adds ability to set notifications settings using the Prometheus conversion API.
The API now supports a new optional header: X-Grafana-Alerting-Notification-Settings which can be used to specify notification settings.
The value of the header is the AlertRuleNotificationSettings structure in JSON:
mimirtool rules load alerts.yaml --extra-headers 'X-Grafana-Alerting-Notification-Settings: {"receiver": "my-webhook", "group_by": ["cluster", "pod"]}'
* add active_time_intervals to route model
* update k8s compat layer
* update notification policies service to validate active time intervals
* update integration tests
* update openapi
* add active time interval to model
* update route generator to include active time interval
* Update storage list and rename methods to handle active intervals
* update api model
* update provisioning and export models
* update ui to allow active timing config
* update i18n
* fix snapshots for ui tests
* run prettier
* Alerting: Active time intervals UI naming (#104402)
* update naming in UI
* update naming in the edit page title
* update translations
* update alerting module
---------
Signed-off-by: Yuri Tseretyan <yuriy.tseretyan@grafana.com>
Co-authored-by: Sonia Aguilar <33540275+soniaAguilarPeiron@users.noreply.github.com>
Co-authored-by: Sonia Aguilar <soniaaguilarpeiron@gmail.com>
* Alerting: Add extended definition to prometheus alert rules api
This adds `isPaused` and `notificationSettings` to the paginated rules api to enable the paginated view of GMA rules.
refactor: make alert rule status and state retrieval extensible
This lets us get status from other sources than the local ruler.
* update swagger spec
* add safety checks in test
* Remove POST config for Grafana Alertmanager
* Delete auth + test for removed path
* Alerting: Remove check for `alertingApiServer` toggle in UI (#103805)
* Remove check for alertingApiServer in UI
* Update tests to no longer care about alertingApiServer
* Add contact points handlers now that we use alertingApiServer all the time
* Fix test broken from removing camelCase for UIDs
---------
Co-authored-by: Tom Ratcliffe <tom.ratcliffe@grafana.com>
* Template editor syntax highlighting when preview is json-like
* Add new template editor language examples, snippets, and functions
* Use updated NewTemplate function
* Add new fields to webhook notifier
- CustomPayload
- ExtraHeaders
* Documentation
* Update grafana/alerting to in-progress PR (needs updating after merge)
* Fix integration test
* Remove docs reference to .Extra template context
No longer exists, was part of a previous iteration
* make update-workspace
* Update grafana/alerting to actual merged commit
API Changes:
- Fixes validation in template CRUD API to be closer to how the running
alertmanager will use the template. Should remove some incorrect
validation errors.
- Adds some missing default placeholder labels to receiver testing that
are used during template testing but missing during receiver testing
Template Preview:
- Replaced basic preview with a readonly CodeEditor for better whitespace
and alignment clarity (also adds support for future syntax highlighting
in template previews for upcoming webhook payload templates)
Template Selector (Receiver Form):
- Refactored to use same components as Template editor for preview.
- Fixed preview to work with multi-definition templates
- Fixed copy to correctly copy the template contents instead of
{{ template "<name>" . }}.
Template Editor:
- Fixed detection of when to display functions vs snippets in multi-line
expressions
What is this feature?
A follow-up for #101184, adds AlertRule.MissingSeriesEvalsToResolve to the APIs.
missing_series_evals_to_resolve must be specified too and it must be > 0.
POST /api/ruler/grafana/api/v1/rules/{folderUID} works in the following way:
If missing_series_evals_to_resolve is not sent or null, the rule keeps its existing value
If missing_series_evals_to_resolve > 0: updates to that value
If missing_series_evals_to_resolve = 0: resets to default (nil).
AlertRule.MissingSeriesEvalsToResolve can't be 0, so I used it to reset
In the Provisioning API, the value is just set if present and > 0. Otherwise it's reset:
PUT to /api/v1/provisioning/alert-rules/{UID}:
If missing_series_evals_to_resolve is nil, it's reset to the default value
If missing_series_evals_to_resolve > 0, it's updated
What is this feature?
This PR introduces a new alert rule configuration option, keep_firing_for (Prometheus documentation).
keep_firing_for prevents alerts from resolving immediately after the alert condition returns to normal. Instead, they transition into a "Recovering" state and are not considered resolved by the Alertmanager. Once the recovery period ends (or after the next evaluation if it is bigger than keep_firing_for), the alert transitions to "Normal" if it doesn't start alerting again:
Before
+----------+ +----------+
| Alerting |---->| Normal |
+----------+ +----------+
-----
After
+----------+ +------------+ +----------+
| Alerting |----->| Recovering |---->| Normal |
+----------+ +------------+ +----------+
Why do we need this feature?
This feature prevents flapping alerts by adding a recovery period. This helps avoid false resolutions caused by brief alert
* add query parameter to existing APIs to control the permanent deletion of rules
* add GUID to gettable rule
* add new endpoint /ruler/grafana/api/v1/trash/rule/guid/{RuleGUID} to delete rules from trash permanently
---------
Signed-off-by: Yuri Tseretyan <yuriy.tseretyan@grafana.com>
When creating Grafana-managed alerts from Prometheus rule definitions with mimirtool or cortextool, the rules are marked as "provisioned" and are not editable in the Grafana UI. This PR allows changing this by providing an extra header: --extra-header="X-Disable-Provenance=true".
When provenance is disabled, we do not keep the original rule definition in YAML, so it is impossible to read it back using the Prometheus conversion API (mimirtool/cortextool). This is intentional because if we did keep it and the rule was later changed in the UI, its Prometheus YAML definition would no longer reflect the latest version of the alert rule, as it would be unchanged.
What is this feature?
Adds target datasource UID to the recording rules so that they write to the same datasource used for alerting rule queries after the import.
Why do we need this feature?
Target datasourse support was added in #101678, and under a feature flag grafanaManagedRecordingRulesDatasources (#101778).
This PR makes the importing process:
Check if the import contains recording rules
Verify both recording rules and the grafanaManagedRecordingRulesDatasources feature flag are enabled
If either check fails, return an error
If both checks pass, create recording rules with the provided datasource UID set as both the query and target datasource
What is this feature?
Allows the creation of alert rules with mimirtool in a specified folder.
Why do we need this feature?
Currently, the APIs for mimirtool create namespaces and rule groups in the root folder without the ability to set a custom folder. For example, it could be a special "Imported" folder, etc.
This PR makes it possible with a special header: mimirtool ... --extra-headers="X-Grafana-Alerting-Folder-UID=123". If it's not present, the root folder is used, otherwise, the specified one is used.
mimirtool does not support nested folder structures, while Grafana allows folder nesting. To keep compatibility, we return only direct child folders of the working folder (as namespaces) with rule groups and rules that are directly in these child folders as if there are no nested folders.
For example, given this folder structure in Grafana:
```
grafana/
├── production/
│ ├── service1/
│ │ └── alerts/
│ └── service2/
└── testing/
└── service3/
```
If the working folder is "grafana":
Only namespaces "production" and "testing" are returned
Only rule groups directly within these folders are included
If the working folder is "production":
- Only namespaces "service1" and "service2" are returned
Only rule groups directly within these folders are included
Initially, Metadata had only the EditorSettings, and HasMetadata was used to understand if the incoming update request had metadata in the body because it could be omitted if it was empty. For example, when the rule is updated via the provisioning API or has only false values. If it was in the request, we used that; if not, we used the metadata from the existing rule from the database. If the rule was updated via the AlertRuleService, we didn't change Metadata at all if the rule already existed.
But now, Metadata also has the Prometheus rule definition, and we always need to update it with the new version of the AlertRuleService when the rule exists in the DB and has the same UID. HasMetadata is renamed to HasEditorSettings to keep the old behaviour only for EditorSettings.
Now, the provisioning API and the conversion API will overwrite everything except EditorSettings with the new data.
What is this feature?
Adds an API endpoint to create alert rules with mimirtool:
- POST /convert/prometheus/config/v1/rules/{NamespaceTitle} - Accepts a single rule group in a Prometheus YAML format and creates or updates a Grafana rule group from it.
The endpoint uses the conversion package from #100224.
Key parts
The API works similarly to the provisioning API. If the rule does not exist, it will be created, otherwise updated. Any rules not present in the new group will be deleted, ensuring the group is fully synchronized with the provided configuration.
Since the API works with namespace titles (folders), the handler automatically creates a folder in the root based on the provided title if it does not exist. It also requires a special header, X-Grafana-Alerting-Datasource-UID. This header specifies which datasource to use for the new rules.
If the rule group's evaluation interval is not specified, it uses the DefaultRuleEvaluationInterval from settings.
* Alerting: Fix Alertmanager configuration updates
Alertmanager configuration updates would behave inconsistently when performing no-op updates with `mysql` as the store.
In particular this bug manifested as a failure to reload the provisioned alertmanager configuration components with no changes to the configuration itself. This would result in a 500 error with mysql store only.
The core issue is that we were relying on the number of rows affected by the update query to determine if the configuration was found in the db or not.
While this behavior works for certain sql dialects, mysql does not return the number of rows matched by the update query but rather the number of rows actually updated.
Also discovered and fixed the mismatched `xorm` tag for the `CreatedAt` field to match the actual column name in the db.
References: https://dev.mysql.com/doc/refman/8.4/en/update.html
When exporting contact-points, mute-timings, and notification policies in the provisioning API, we need to escape the `$` character which is used in interpolation by file provisioning.
Follow up to #97985
