85 Commits

Author SHA1 Message Date
bd87b46b15 Alerting: Improve test coverage for ConditionsCmd (#58603) 2022-11-11 09:27:35 +00:00
acfa066ba2 Alerting: Change TestConditionsCmd to assert on mathexp.Results (#58434) 2022-11-09 16:07:43 +00:00
fdeefaee42 Alerting: Reorder tests in classic_test.go (#58425) 2022-11-09 09:13:47 +00:00
aa69a8463f Revert "Alerting: Fix mathexp.NoData in ConditionsCmd (#56812)" (#58423)
This reverts commit 5fa0936b7e6ee8346f7b821a5076269baac506d9.
2022-11-08 13:35:58 +00:00
5fa0936b7e Alerting: Fix mathexp.NoData in ConditionsCmd (#56812)
This commit fixes an issue where mathexp.NoData would return an error
in ConditionsCmd (Classic Condition) instead of no data. It further
refactors the Execute method to make it easier to understand.
2022-11-08 11:41:57 +00:00
0a4121cef8 Alerting: Contextual log provider for rule key (#57476)
* create contextual log context provider
* use contextual provider in scheduler
* init logger in the package
* use context for log context
* use context in state manager
2022-10-26 19:16:02 -04:00
2d20c8db7b Chore: Expression engine to support relative time range (#57474)
* make TimeRange interface and add relative range
* make Execute methods support the current time
* update resample to support relative time range
* update DSNode to support relative time range
* update query service to create queries with absolute time
* make alerting evaluator create relative time ranges
2022-10-26 16:13:58 -04:00
245c1ee3d3 Chore: Add last downsampling function to Resample expression (#57379) 2022-10-21 10:03:43 -04:00
004bb7689d Alerting: Fix mathexp.NoData for ConditionsCmd (#56816) 2022-10-12 17:34:28 +01:00
67d93ceea0 Alerting: Add doc comments to classic.go (#56724) 2022-10-12 10:55:48 +01:00
9aa61ddd0e Alerting: Add threshold expression (#55102) 2022-09-26 16:05:44 +02:00
2d38664fe6 Alerting: Improve validation of query and expressions on rule submit (#53258)
* Improve error messages of server-side expression 
* move validation of alert queries and a condition to eval package
2022-09-21 15:14:11 -04:00
7d20766ae9 Alerting: Fix mathexp.NoData cannot be reduced (#55347)
This commit fixes a bug where queries from datasources such as InfluxDB that returned no data would not create a DatasourceNoData alert, but instead an error "can only reduce type series, got type noData".
2022-09-21 10:01:51 +01:00
b287047052 Chore: Upgrade Go to 1.19.1 (#54902)
* WIP

* Set public_suffix to a pre Ruby 2.6 version

* we don't need to install python

* Stretch->Buster

* Bump versions in lib.star

* Manually update linter

Sort of messy, but the .mod-file need to contain all dependencies that
use 1.16+ features, otherwise they're assumed to be compiled with
-lang=go1.16 and cannot access generics et al.

Bingo doesn't seem to understand that, but it's possible to manually
update things to get Bingo happy.

* undo reformatting

* Various lint improvements

* More from the linter

* goimports -w ./pkg/

* Disable gocritic

* Add/modify linter exceptions

* lint + flatten nested list

Go 1.19 doesn't support nested lists, and there wasn't an obvious workaround.
https://go.dev/doc/comment#lists
2022-09-12 12:03:49 +02:00
05fd7eb047 SSE: Add noData type (#51973)
When there is a single frame with no fields (e.g. splunk datasource) SSE errors when trying to figure out the data type. This frame needs to exist since this is where the executedQueryString metadata exists.

This adds a new return type to SSE to represent no data, so the original frame with its metadata can still be maintained.
2022-07-14 09:18:12 -04:00
9aa6ce2a50 Datasource: Propagate datasource secret decryption errors to the frontend (#52068)
* update decrypt secrets function signature and add secrets error handling

* remove a couple instances of unnecessary logging since errors are properly handled now

* add unit test

* fix linting issues
2022-07-13 09:27:03 -04:00
945f015770 backend/datasources: move datasources models into the datasources service package (#51267)
* backend/datasources: move datasources models into the datasources service pkg
2022-06-27 12:23:15 -04:00
3c3039f5b3 Chore: Remove Wrap (#50048)
* Chore: Remove Wrap and Wrapf

* Fix: Add error check
2022-06-03 09:24:24 +02:00
01ef899753 SSE/Alerting: Support prom instant vector responses (#44865)
* SSE/Alerting: (Draft) Support prom instant vector responses
fixes #35663
* reduce\classic expressions to handle mathexp.Number
* use Notice for warning

Co-authored-by: Yuriy Tseretyan <yuriy.tseretyan@grafana.com>
2022-05-23 10:08:14 -04:00
a367ad730c Secrets: Implement basic unified secret store service (#45804)
* wip: Implement kvstore for secrets

* wip: Refactor kvstore for secrets

* wip: Add format key function to secrets kvstore sql

* wip: Add migration for secrets kvstore

* Remove unused Key field from secrets kvstore

* Remove secret values from debug logs

* Integrate unified secrets with datasources

* Fix minor issues and tests for kvstore

* Create test service helper for secret store

* Remove encryption tests from datasources

* Move secret operations after datasources

* Fix datasource proxy tests

* Fix legacy data tests

* Add Name to all delete data source commands

* Implement decryption cache on sql secret store

* Fix minor issue with cache and tests

* Use secret type on secret store datasource operations

* Add comments to make create and update clear

* Rename itemFound variable to isFound

* Improve secret deletion and cache management

* Add base64 encoding to sql secret store

* Move secret retrieval to decrypted values function

* Refactor decrypt secure json data functions

* Fix expr tests

* Fix datasource tests

* Fix plugin proxy tests

* Fix query tests

* Fix metrics api tests

* Remove unused fake secrets service from query tests

* Add rename function to secret store

* Add check for error renaming secret

* Remove bus from tests to fix merge conflicts

* Add background secrets migration to datasources

* Get datasource secure json fields from secrets

* Move migration to secret store

* Revert "Move migration to secret store"

This reverts commit 7c3f872072e9aff601fb9d639127d468c03f97ef.

* Add secret service to datasource service on tests

* Fix datasource tests

* Remove merge conflict on wire

* Add ctx to data source http transport on prometheus stats collector

* Add ctx to data source http transport on stats collector test
2022-04-25 13:57:45 -03:00
8f6877e12a chore: remove all remaining uses of golang.org/x/net/context (#47564)
* chore: remove all remaining uses of  golang.org/x/net/context

This PR finishes the work started in #47532, replacing all calls to that package with the stdlib context and using http.NewRequestWithContext to include the context where necessary.

Bonus: small formatting fixes to goimports in these files.

closes #44178

* tweak: use context.Background in favor of TODO for tests
2022-04-11 14:20:10 -04:00
18e93c7077 Chore: Remove bus (#47511)
* Chore: Remove bus

* remove unused const
2022-04-08 16:15:06 +02:00
cab509fbe8 SSE: Keep FieldConfig for data source queries (#46328)
for #46325
2022-03-10 10:03:26 -05:00
a578cf0f7c SSE: Fix NoData when some series were no data but others not (#45867)
Co-authored-by: Santiago <santiagohernandez.1997@gmail.com>
2022-02-24 21:01:04 -03:00
040ce40113 SSE: Mode to drop NaN/Inf/Null in Reduction operations (#43583)
Co-authored-by: Yuriy Tseretyan <yuriy.tseretyan@grafana.com>
Co-authored-by: gillesdemey <gilles.de.mey@gmail.com>
2022-02-02 14:50:44 +01:00
1bed84ea56 Fix InfluxDB unhealthy alerts and false "no data"s (#43441)
* check for TimeSeriesTypeNot to fix InfluxDB alerts

* log a warning when a data frame is ignored
2022-01-27 10:53:27 -03:00
9fc0aee02b Feature/mathandreduce (#41608)
* Added new math functions: round, ceil, floor 
* Added a new reduce function: last.
2022-01-21 13:15:50 -05:00
f17fb76b5e SSE: Add is_nan() and other like functions and allow '_' in func names (#43555)
* SSE: Add is_nan() and allow '_' in func names

is_null() infn(), is_inf(), is_number()
2021-12-29 11:40:52 -05:00
7936c4c522 Rename AddHandlerCtx to AddHandler (#43557) 2021-12-28 16:08:07 +01:00
2754e4fdf0 Expressions: use datasource model from the query (#41376)
* refactor datasource loading

* refactor datasource loading

* pass uid

* use dscache in alerting to get DS

* remove expr/translate pacakge

* remove dup injection entry

* fix DS type on metrics endpoint, remove SQL DS lookup inside SSE

* update test and adapter

* comment fix

* Make eval run as admin when getting datasource info

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* fmt and comment

* remove unncessary/redundant code

Co-authored-by: Kyle Brandt <kyle@grafana.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Santiago <santiagohernandez.1997@gmail.com>
2021-12-16 13:51:46 -03:00
8927a3ca20 Chore: Query endpoint refactor (#41637)
Get rid of using legacydata.RequestHandler in HTTPServer, /api/tsdb/query and pkg/expr 
with the goal of deprecating /api/tsdb/query and remove it completely eventually. This is 
the first step of cleaning up the HTTP API query endpoint.

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
2021-11-29 14:21:54 +01:00
4014891971 Encryption: Cleanup and add logging (#42084)
* Encryption: Add more logs

* Add logging and checks

* Removed unused methods from secrets service

* Refactor and update tests

* Address review feedback
2021-11-24 15:01:44 +02:00
708bdc80cb Alerting: Support Unwrap for QueryError in expr package (#41743) 2021-11-17 10:07:24 +00:00
2319c52c85 Alerting: fix a bug where the metric in the evaluation string was not correctly populated (#41731)
Instead of using the frame name as the series name, we now use the name of the series value column if the frame name is not set. Queries to some datasources may end up with an empty frame name, which leads to an empty series name. As the series name is used in the frontend to display which metrics were queried etc. it shouldn't be empty.
2021-11-17 11:00:59 +01:00
bc60ae3c66 Encryption: Refactor secrets service (#41771)
* Refactor kmsproviders pkg

* Update tests

* Fix linting

Co-authored-by: Joan López de la Franca Beltran <joanjan14@gmail.com>
2021-11-17 11:52:45 +02:00
543b1a7e39 Alerting: Add QueryError to expr package (#41737) 2021-11-16 13:42:22 +00:00
44837fc592 Replace encryption.Service usages by secrets.Service (#41625)
* Replace encryption.Service by secrets.Service on expr.Service

* Replace encryption.Service by secrets.Service on live pkg

* Rename encryption.Service to encryption.Internal to clarify it must be not used
2021-11-12 12:16:39 +01:00
baab021fec Chore: Refactor usage of legacy data contracts (#41218)
Refactor usage of legacy data contracts. Moves legacy data contracts 
to pkg/tsdb/legacydata package.
Refactor pkg/expr to be a proper service/dependency that can be provided 
to wire to remove some unneeded dependencies to SSE in ngalert and other places.
Refactor pkg/expr to not use the legacydata,RequestHandler and use 
backend.QueryDataHandler instead.
2021-11-10 11:52:16 +01:00
3489721ed6 api/ds/query: simplify data sources lookup for queries and expressions (#41172) 2021-11-05 08:12:55 -07:00
469a5e4a85 Comment out "Queries with new datasource ref object" test (#41187) 2021-11-02 13:09:51 +02:00
7319efe077 Dashboard: replace datasource name with a reference object (#33817)
Co-authored-by: Torkel Ödegaard <torkel@grafana.com>
Co-authored-by: Elfo404 <me@giordanoricci.com>
2021-10-29 10:57:24 -07:00
e4297006f6 Chore: Propagate context for data source provisioning (#40235)
* context all the things

* apply feedback

* rollback some alerting changes

* rollback some alerting changes #2

* more rollbacks

* more rollbacks #2

* more rollbacks #3

* more rollbacks #4

* fix integration test

* add missing context

* add missing and remove incorrect dispatch
2021-10-18 17:06:19 +02:00
64c8d32fe7 Use sdk pkg for gtime (#39354) 2021-09-21 13:08:52 +02:00
202f295e51 Chore: Fix typo in nodes.go (#38636)
commmon -> common
2021-08-27 14:57:41 +02:00
78596a6756 Migrate to Wire for dependency injection (#32289)
Fixes #30144

Co-authored-by: dsotirakis <sotirakis.dim@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Leon Sorokin <leeoniya@gmail.com>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: spinillos <selenepinillos@gmail.com>
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
2021-08-25 15:11:22 +02:00
3e124c854e Alerting: interpolate variables when creating alert rule from dashboard panel (#37201) 2021-08-10 10:59:48 +03:00
b6a7156300 Alerting: Handle marshaling Inf values (#36947)
* SSE: change EvalMatch json value encoding to string
* update MarshalJSON on null.Float to handle inf as null
fixes #36424
2021-07-21 10:04:40 -04:00
ea2ba06b93 CloudWatch/Logs: Fix log alerts in new unified alerting (#36558)
* Pass FromAlert header from new alerting

* Add better error messages
2021-07-09 13:43:22 +02:00
abe35c8c01 Alerting: Add error recovery during rule evaluations (#35450)
* Alerting: Eval recovery after query failure

* Apply suggestions from code review
2021-06-15 19:30:21 +03:00
c997f646cc Alerting: Fix alert migration RefID generation (#35667)
* Alerting: Fix alert migration RefID generation

if the alert has more than 26 conditions

* Remove TODO comment

* Make similar change under pkg/expr
2021-06-14 22:13:45 +03:00