114 Commits

Author SHA1 Message Date
cbbbe2e6f6 SSE: DSNode to update result with names to make each value identifiable by labels (only Graphite and TestData) (#71246)
* introduce a function checkIfSeriesNeedToBeFixed to scan all value fields in the response and provide a function that updates Series so they can be uniquely identifiable. Only Graphite and TestData are checked.

* update `convertDataFramesToResults` to run this function and provide it to WideToMany
* update WideToMany to run the fix function if it is not nil
2023-07-20 14:44:12 -04:00
a8577c21ba Plugins: Migrate PluginStore mock to pre-existing fakes package (#71664)
* migrate to existing fakes package

* fix imports
2023-07-17 10:21:44 +00:00
541bfe636d SSE: Support for ML query node (#69963)
* introduce a new node-type ML and implement a command outlier that uses ML plugin as a source of data.
* add feature flag mlExpressions that guards the feature
2023-07-13 20:37:50 +03:00
9e76d6372c SSE: improve error handling in DSNode (#71468)
* add data source uid to QueryError

* add error and datasource uid to tracing

* split queryDataResponseToResults to two functions one to extract frame from the result, and another to convert it

* propagate logger with context to convertDataFramesToResults
2023-07-12 14:59:02 -04:00
842f33580e SSE: Add functions that determine NodeType by UID and construct a data source struct from NodeType (#70106)
* add NodeTypeFromDatasourceUID and DataSourceModelFromNodeType()
* deprecate expr.DataSourceModel
* replace usages of IsDataSource to NodeTypeFromDatasourceUID 
* replace usages of DataSourceModel to DataSourceModelFromNodeType()
2023-06-16 13:05:06 -04:00
cc72fe17d5 SSE: Refactor DSNode and extract function to convert backend response to mathexp.Results (#70098) 2023-06-15 09:20:08 -04:00
624777258b Plugins: Refactor creation of plugin context to dedicated service (#66451)
* first pass

* fix tests

* return errs

* change signature

* tidy

* delete unnecessary fields from test

* tidy

* fix tests

* simplify

* separate error check in API

* apply nits
2023-06-08 13:59:51 +02:00
350de3f3bf Expressions/threshold: Fix incorrect thresholds args length (#66859) 2023-04-20 05:42:34 -04:00
840fb32ad8 SSE: (Instrumentation) Add Tracing (#66700)
spans are prefixed `SSE.`
2023-04-18 08:04:51 -04:00
2f13c851e4 SSE: (Chore/Instrumentation) Add ds_queries_total metric and move met… (#66695)
* SSE: (Chore/Instrumentation) Add ds_queries_total metric and move metrics to service
2023-04-17 16:12:44 -07:00
e78be44e1a SSE: Dataplane Compliance (#65927)
Takes a specific code path for data that identifies itself as dataplane instead of "guessing" what the data is.

The data must identify itself by being in the dataplane by having both the following frame metadata properties:

- TypeVersion property that is greater than 0.0
- 'Type' property

The flag is disableSSEDataplane and disables this functionality and uses the old code for all queries regardless.

See https://github.com/grafana/grafana-plugin-sdk-go/blob/main/data/contract_docs/contract.md for dataplane details.
2023-04-12 12:24:34 -04:00
972e611f76 CloudWatch Logs: Revert "Queries in an expression should run synchronously (#64443)" (#65036)
Revert "CloudWatch Logs: Queries in an expression should run synchronously (#64443)"

This reverts commit 74436d31de094a7cdd04de7ed7552e77e0f467af.
2023-03-20 15:54:30 +01:00
74436d31de CloudWatch Logs: Queries in an expression should run synchronously (#64443) 2023-03-13 13:31:46 -04:00
94f39e69a3 Plugins: Migrate plugincontext, adapters and pluginsettings to pkg/services/pluginsintegration package (#64154)
* migrate plugincontext, adapter and pluginsettings

* add to CODEOWNERS

* fix imports

* fix CODEOWNERS

* take pluginsettings

* migrate wire stuff
2023-03-07 11:22:30 -05:00
0bdb105df2 Chore: Remove xorcare/pointer dependency (#63900)
* Chore: remove pointer dependency

* fix type casts

* deprecate xorcare/pointer library in linter

* rooky mistake
2023-03-06 05:23:15 -05:00
23c27cffb3 Chore: Rename Id to ID in alerting models (#62777)
* Chore: Rename Id to ID in alerting models

* Add xorm tags for datasource

* Add xorm tag for uid
2023-02-02 17:22:43 +01:00
f8ec35e643 Chore: Fix goimports grouping (#62427)
* fix goimports order

* fix goimports order
2023-01-30 09:38:51 +01:00
e5920c211e Chore: Fix random indices for slices in test files (#61884)
* Fix random indices for slices in test files

* Empty commit
2023-01-24 15:07:37 -03:00
c0913ce718 SSE: Fix math expression to support NoData results (#61721)
* update perFloat to support NoData
* update union to correctly handle no-data
2023-01-20 09:53:03 -05:00
5e8866ed5a SSE: DSNode to parse only one response (#61644)
* do not iterate through responses
* log type of the response in all exit cases
2023-01-18 13:06:10 -05:00
ef6d73e575 SSE: Resample command to support NoData (#61708)
* add test for nil value
2023-01-18 12:39:38 -05:00
402dc5e4d6 Alerting: Redo refactoring from reverted fix in #56812 (#61051)
This pull request re-applies the refactoring of ConditionsCmd from a
reverted fix #56812 for mathexp.noData. It does not add the fix, or
tests for the fix, because those were added in #56816. We use the
additional test coverage added in #56816 and #58650 to avoid the
reoccurrence of regressions that caused us to revert #56812 the
first time.
2023-01-09 17:01:19 +00:00
6dbe3b555f Plugins: Refactor forward of cookies, OAuth token and header modifications by introducing client middlewares (#58132)
Adding support for backend plugin client middlewares. This allows headers in outgoing 
backend plugin and HTTP requests to be modified using client middlewares.

The following client middlewares added:
Forward cookies: Will forward incoming HTTP request Cookies to outgoing plugins.Client 
and HTTP requests if the datasource has enabled forwarding of cookies (keepCookies).
Forward OAuth token: Will set OAuth token headers on outgoing plugins.Client and HTTP 
requests if the datasource has enabled Forward OAuth Identity (oauthPassThru).
Clear auth headers: Will clear any outgoing HTTP headers that was part of the incoming 
HTTP request and used when authenticating to Grafana.
The current suggested way to register client middlewares is to have a separate package, 
pluginsintegration, responsible for bootstrap/instantiate the backend plugin client with 
middlewares and/or longer term bootstrap/instantiate plugin management. 

Fixes #54135
Related to #47734
Related to #57870
Related to #41623
Related to #57065
2022-12-01 19:08:36 +01:00
110fdf4da9 Alerting: Fix ConditionsCmd behavior when last is No Data (#58953) 2022-11-28 21:56:22 +00:00
5623b5afaf SSE: Make sure to forward headers, user and cookies/OAuth token (#58897)
Fixes #58793 and Fixes https://github.com/grafana/azure-data-explorer-datasource/issues/513

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-11-28 07:40:06 -05:00
c9286ff794 Alerting: Remove comments for fixed behavior (#58951) 2022-11-23 15:37:15 +01:00
b77c3946a5 Alerting: Fix ConditionsCmd No Data for "has no value" (#58634)
This commit fixes a bug where ConditionsCmd returns No Data even when
the condition checks for "has no value". It should return 1 with a nil
match.
2022-11-18 09:28:21 +00:00
48c34d310c Alerting: Add tests that check current No Data behaviour with two conditions (#58650) 2022-11-18 09:04:43 +00:00
1953d473c0 SSE: Keep value name from numeric table (#58831)
fixes #48868
2022-11-16 10:39:28 -05:00
bd87b46b15 Alerting: Improve test coverage for ConditionsCmd (#58603) 2022-11-11 09:27:35 +00:00
acfa066ba2 Alerting: Change TestConditionsCmd to assert on mathexp.Results (#58434) 2022-11-09 16:07:43 +00:00
fdeefaee42 Alerting: Reorder tests in classic_test.go (#58425) 2022-11-09 09:13:47 +00:00
aa69a8463f Revert "Alerting: Fix mathexp.NoData in ConditionsCmd (#56812)" (#58423)
This reverts commit 5fa0936b7e6ee8346f7b821a5076269baac506d9.
2022-11-08 13:35:58 +00:00
5fa0936b7e Alerting: Fix mathexp.NoData in ConditionsCmd (#56812)
This commit fixes an issue where mathexp.NoData would return an error
in ConditionsCmd (Classic Condition) instead of no data. It further
refactors the Execute method to make it easier to understand.
2022-11-08 11:41:57 +00:00
0a4121cef8 Alerting: Contextual log provider for rule key (#57476)
* create contextual log context provider
* use contextual provider in scheduler
* init logger in the package
* use context for log context
* use context in state manager
2022-10-26 19:16:02 -04:00
2d20c8db7b Chore: Expression engine to support relative time range (#57474)
* make TimeRange interface and add relative range
* make Execute methods support the current time
* update resample to support relative time range
* update DSNode to support relative time range
* update query service to create queries with absolute time
* make alerting evaluator create relative time ranges
2022-10-26 16:13:58 -04:00
245c1ee3d3 Chore: Add last downsampling function to Resample expression (#57379) 2022-10-21 10:03:43 -04:00
004bb7689d Alerting: Fix mathexp.NoData for ConditionsCmd (#56816) 2022-10-12 17:34:28 +01:00
67d93ceea0 Alerting: Add doc comments to classic.go (#56724) 2022-10-12 10:55:48 +01:00
9aa61ddd0e Alerting: Add threshold expression (#55102) 2022-09-26 16:05:44 +02:00
2d38664fe6 Alerting: Improve validation of query and expressions on rule submit (#53258)
* Improve error messages of server-side expression 
* move validation of alert queries and a condition to eval package
2022-09-21 15:14:11 -04:00
7d20766ae9 Alerting: Fix mathexp.NoData cannot be reduced (#55347)
This commit fixes a bug where queries from datasources such as InfluxDB that returned no data would not create a DatasourceNoData alert, but instead an error "can only reduce type series, got type noData".
2022-09-21 10:01:51 +01:00
b287047052 Chore: Upgrade Go to 1.19.1 (#54902)
* WIP

* Set public_suffix to a pre Ruby 2.6 version

* we don't need to install python

* Stretch->Buster

* Bump versions in lib.star

* Manually update linter

Sort of messy, but the .mod-file need to contain all dependencies that
use 1.16+ features, otherwise they're assumed to be compiled with
-lang=go1.16 and cannot access generics et al.

Bingo doesn't seem to understand that, but it's possible to manually
update things to get Bingo happy.

* undo reformatting

* Various lint improvements

* More from the linter

* goimports -w ./pkg/

* Disable gocritic

* Add/modify linter exceptions

* lint + flatten nested list

Go 1.19 doesn't support nested lists, and there wasn't an obvious workaround.
https://go.dev/doc/comment#lists
2022-09-12 12:03:49 +02:00
05fd7eb047 SSE: Add noData type (#51973)
When there is a single frame with no fields (e.g. splunk datasource) SSE errors when trying to figure out the data type. This frame needs to exist since this is where the executedQueryString metadata exists.

This adds a new return type to SSE to represent no data, so the original frame with its metadata can still be maintained.
2022-07-14 09:18:12 -04:00
9aa6ce2a50 Datasource: Propagate datasource secret decryption errors to the frontend (#52068)
* update decrypt secrets function signature and add secrets error handling

* remove a couple instances of unnecessary logging since errors are properly handled now

* add unit test

* fix linting issues
2022-07-13 09:27:03 -04:00
945f015770 backend/datasources: move datasources models into the datasources service package (#51267)
* backend/datasources: move datasources models into the datasources service pkg
2022-06-27 12:23:15 -04:00
3c3039f5b3 Chore: Remove Wrap (#50048)
* Chore: Remove Wrap and Wrapf

* Fix: Add error check
2022-06-03 09:24:24 +02:00
01ef899753 SSE/Alerting: Support prom instant vector responses (#44865)
* SSE/Alerting: (Draft) Support prom instant vector responses
fixes #35663
* reduce\classic expressions to handle mathexp.Number
* use Notice for warning

Co-authored-by: Yuriy Tseretyan <yuriy.tseretyan@grafana.com>
2022-05-23 10:08:14 -04:00
a367ad730c Secrets: Implement basic unified secret store service (#45804)
* wip: Implement kvstore for secrets

* wip: Refactor kvstore for secrets

* wip: Add format key function to secrets kvstore sql

* wip: Add migration for secrets kvstore

* Remove unused Key field from secrets kvstore

* Remove secret values from debug logs

* Integrate unified secrets with datasources

* Fix minor issues and tests for kvstore

* Create test service helper for secret store

* Remove encryption tests from datasources

* Move secret operations after datasources

* Fix datasource proxy tests

* Fix legacy data tests

* Add Name to all delete data source commands

* Implement decryption cache on sql secret store

* Fix minor issue with cache and tests

* Use secret type on secret store datasource operations

* Add comments to make create and update clear

* Rename itemFound variable to isFound

* Improve secret deletion and cache management

* Add base64 encoding to sql secret store

* Move secret retrieval to decrypted values function

* Refactor decrypt secure json data functions

* Fix expr tests

* Fix datasource tests

* Fix plugin proxy tests

* Fix query tests

* Fix metrics api tests

* Remove unused fake secrets service from query tests

* Add rename function to secret store

* Add check for error renaming secret

* Remove bus from tests to fix merge conflicts

* Add background secrets migration to datasources

* Get datasource secure json fields from secrets

* Move migration to secret store

* Revert "Move migration to secret store"

This reverts commit 7c3f872072e9aff601fb9d639127d468c03f97ef.

* Add secret service to datasource service on tests

* Fix datasource tests

* Remove merge conflict on wire

* Add ctx to data source http transport on prometheus stats collector

* Add ctx to data source http transport on stats collector test
2022-04-25 13:57:45 -03:00
8f6877e12a chore: remove all remaining uses of golang.org/x/net/context (#47564)
* chore: remove all remaining uses of  golang.org/x/net/context

This PR finishes the work started in #47532, replacing all calls to that package with the stdlib context and using http.NewRequestWithContext to include the context where necessary.

Bonus: small formatting fixes to goimports in these files.

closes #44178

* tweak: use context.Background in favor of TODO for tests
2022-04-11 14:20:10 -04:00