139 Commits

Author SHA1 Message Date
f23f50f58d Expressions: Add model struct for the query types (not map[string]any) (#82745) 2024-02-17 02:59:11 +02:00
3f30cbf91c DataSources: Add datasource fetching + querying interface (#80749)
* first pass

* separate oss + enterprise

* tidy things up

* add ctx

* fix tests

* use standalone svcs

* mv plugin context provide

* fix wire

* fix import
2024-01-19 15:56:52 +01:00
bf8be46e6f SSE: Add utility methods for HysteresisCommand (#79157)
* add GetCommandsFromPipeline
* refactor method GetCommandType to func GetExpressionCommandType
* add function to create fingerprint frames
* add function to determine whether raw query represents a hysteresis command and a function to patch it with loaded metrics
2023-12-11 22:40:31 +02:00
cb0a88a027 Plugins: Backend: Skip host environment variables (#77858)
* Add pluginsSkipHostEnvVars feature flag

* Set go-plugin's SkipHostEnvVar depending on feature flags

* add missing file

* Re-generate feature flags

* Add allowedHostEnvVarNames

* Fix feature toggles not being passed to plugin context service's plugin env vars

* Fix tests

* PR review feedback: Use cfg.Features

* Fix tests

* PR review feedback: removed DefaultProviderWithFeatures

* merge with master

* fix tests

* use features.IsEnabledGlobally
2023-11-15 18:09:14 +01:00
f69fd3726b FeatureToggles: Add context and and an explicit global check (#78081) 2023-11-14 12:50:27 -08:00
5d5f8dfc52 Chore: Upgrade Go to 1.21.3 (#77304) 2023-11-01 09:17:38 -07:00
810fbc3327 SSE: Support hysteresis threshold expression (#70998)
* extend threshold command with second evaluator called `unloadEvaluator` 
* Introduce a new expression command Hysteresis and update Threshold unmarshaller to create the HysteresisCommand if the second eval
* add feature flag `recoveryThreshold`
* update unmarshal threshold command to not re-marshall because it breaks frame definition by shuffling the schema and data fields
2023-10-10 17:51:50 +03:00
e4c1a7a141 Tracing: Standardize on otel tracing (#75528) 2023-10-03 14:54:20 +02:00
0128d0403f Tracing: Use tracing.InitializeTracerForTest (#75479) 2023-09-27 09:51:57 +02:00
ece94b1e01 Chore: Remove plugincontext.ErrPluginNotFound (#74997) 2023-09-25 13:10:47 +03:00
7fca1bde54 Plugins: Set grafana config, plugin version and user agent on plugin requests (#75171)
* first pass

* fixup

* remove test line

* fix tests

* use new fields

* fix imports + formatting

* fix tests

* rollback changes

* undo whitespace

* apply pr feedback
2023-09-21 11:33:31 +02:00
40c12c17bf SSE: Reduce to add warning notice to only the first result (#74940) 2023-09-15 11:07:17 -04:00
68eda133e4 SSE: Reduce to apply Mode to instant vector (mathexp.Number) (#74859) 2023-09-14 12:26:00 -04:00
35e488b22b SSE: Localize/Contain Errors within an Expression (#73163)
Changes SSE to not always fail all queries when one fails. Now only the query itself, and nodes that depend on it will error.
---------

Co-authored-by: Gilles De Mey <gilles.de.mey@gmail.com>
2023-09-13 13:58:16 -04:00
e855efb13d Plugins: Move store and plugin dto to pluginsintegration (#74655)
move store and plugin dto
2023-09-11 13:59:24 +02:00
5cc737bb24 SSE: Put data source query grouping behind feature flag (#74551)
change was originally merged in commit: 720d716 via PR: https://github.com/grafana/grafana/pull/72935 with no flag
flag is: sseGroupByDatasource
2023-09-07 23:02:07 +03:00
025b2f3011 Chore: use any rather than interface{} (#74066) 2023-08-30 18:46:47 +03:00
1b8e9b51b2 Replace signed in user for identity.requester (#74048)
* Make identity.Requester available at Context

* Clean pkg/services/guardian/guardian.go

* Clean guardian provider and guardian AC

* Clean pkg/api/team.go

* Clean ctxhandler, datasources, plugin and live

* Clean dashboards and guardian

* Implement NewUserDisplayDTOFromRequester

* Change status code numbers for http constants

* Upgrade signature of ngalert services

* log parsing errors instead of throwing error
2023-08-30 16:51:18 +02:00
Jo
a307582212 Revert "Replace signed in user for identity.requester (#73750)" (#73962)
This reverts commit 9b9c9e83dc7131c7abdb7ea538af39feaa8f9596.
2023-08-28 21:05:59 +02:00
9b9c9e83dc Replace signed in user for identity.requester (#73750)
* Make identity.Requester available at Context

* Clean pkg/services/guardian/guardian.go

* Clean guardian provider and guardian AC

* Clean pkg/api/team.go

* Clean ctxhandler, datasources, plugin and live

* Question: what to do with the UserDisplayDTO?

* Clean dashboards and guardian

* Remove identity.Requester from ReqContext

* Implement NewUserDisplayDTOFromRequester

* Fix tests

* Change status code numbers for http constants

* Upgrade signature of ngalert services

* log parsing errors instead of throwing error

* Fix tests and add logs

* linting
2023-08-28 12:04:36 -05:00
040b7d2571 Chore: Add errutils helpers (#73577)
Add helpers for the errutil package in favor of errutil.NewBase.
2023-08-22 12:52:24 +02:00
720d716e45 SSE: Group data source node execution by data source (#72935)
Execute all queries to the same datasource in a single request.

Uses the query index and the graph node ID index, and then a stable dependency graph sort based on node input index number in attempt to keep the original query order intact.
2023-08-18 07:49:59 -04:00
d0ad4fcd0a SSE: Warn on dropped items in Union in Math Operation (#72682) 2023-08-03 14:23:18 -04:00
ceb694f165 SSE: (Instrumentation/Chore) Add datasource_type label to grafana_sse (#72370)
SSE: (Instrumentation/Chore) Add datasource_type label to the grafana_sse_ds_queries_total metric
2023-07-26 10:51:02 -04:00
1df4d332c9 SSE: Use errutil to show better error messages in prod (#71658)
- include public message
- propagate data source query errors so they are shown as well to which fixes #70026
2023-07-21 06:38:29 -04:00
cbbbe2e6f6 SSE: DSNode to update result with names to make each value identifiable by labels (only Graphite and TestData) (#71246)
* introduce a function checkIfSeriesNeedToBeFixed to scan all value fields in the response and provide a function that updates Series so they can be uniquely identifiable. Only Graphite and TestData are checked.

* update `convertDataFramesToResults` to run this function and provide it to WideToMany
* update WideToMany to run the fix function if it is not nil
2023-07-20 14:44:12 -04:00
a8577c21ba Plugins: Migrate PluginStore mock to pre-existing fakes package (#71664)
* migrate to existing fakes package

* fix imports
2023-07-17 10:21:44 +00:00
541bfe636d SSE: Support for ML query node (#69963)
* introduce a new node-type ML and implement a command outlier that uses ML plugin as a source of data.
* add feature flag mlExpressions that guards the feature
2023-07-13 20:37:50 +03:00
9e76d6372c SSE: improve error handling in DSNode (#71468)
* add data source uid to QueryError

* add error and datasource uid to tracing

* split queryDataResponseToResults to two functions one to extract frame from the result, and another to convert it

* propagate logger with context to convertDataFramesToResults
2023-07-12 14:59:02 -04:00
842f33580e SSE: Add functions that determine NodeType by UID and construct a data source struct from NodeType (#70106)
* add NodeTypeFromDatasourceUID and DataSourceModelFromNodeType()
* deprecate expr.DataSourceModel
* replace usages of IsDataSource to NodeTypeFromDatasourceUID 
* replace usages of DataSourceModel to DataSourceModelFromNodeType()
2023-06-16 13:05:06 -04:00
cc72fe17d5 SSE: Refactor DSNode and extract function to convert backend response to mathexp.Results (#70098) 2023-06-15 09:20:08 -04:00
624777258b Plugins: Refactor creation of plugin context to dedicated service (#66451)
* first pass

* fix tests

* return errs

* change signature

* tidy

* delete unnecessary fields from test

* tidy

* fix tests

* simplify

* separate error check in API

* apply nits
2023-06-08 13:59:51 +02:00
350de3f3bf Expressions/threshold: Fix incorrect thresholds args length (#66859) 2023-04-20 05:42:34 -04:00
840fb32ad8 SSE: (Instrumentation) Add Tracing (#66700)
spans are prefixed `SSE.`
2023-04-18 08:04:51 -04:00
2f13c851e4 SSE: (Chore/Instrumentation) Add ds_queries_total metric and move met… (#66695)
* SSE: (Chore/Instrumentation) Add ds_queries_total metric and move metrics to service
2023-04-17 16:12:44 -07:00
e78be44e1a SSE: Dataplane Compliance (#65927)
Takes a specific code path for data that identifies itself as dataplane instead of "guessing" what the data is.

The data must identify itself by being in the dataplane by having both the following frame metadata properties:

- TypeVersion property that is greater than 0.0
- 'Type' property

The flag is disableSSEDataplane and disables this functionality and uses the old code for all queries regardless.

See https://github.com/grafana/grafana-plugin-sdk-go/blob/main/data/contract_docs/contract.md for dataplane details.
2023-04-12 12:24:34 -04:00
972e611f76 CloudWatch Logs: Revert "Queries in an expression should run synchronously (#64443)" (#65036)
Revert "CloudWatch Logs: Queries in an expression should run synchronously (#64443)"

This reverts commit 74436d31de094a7cdd04de7ed7552e77e0f467af.
2023-03-20 15:54:30 +01:00
74436d31de CloudWatch Logs: Queries in an expression should run synchronously (#64443) 2023-03-13 13:31:46 -04:00
94f39e69a3 Plugins: Migrate plugincontext, adapters and pluginsettings to pkg/services/pluginsintegration package (#64154)
* migrate plugincontext, adapter and pluginsettings

* add to CODEOWNERS

* fix imports

* fix CODEOWNERS

* take pluginsettings

* migrate wire stuff
2023-03-07 11:22:30 -05:00
0bdb105df2 Chore: Remove xorcare/pointer dependency (#63900)
* Chore: remove pointer dependency

* fix type casts

* deprecate xorcare/pointer library in linter

* rooky mistake
2023-03-06 05:23:15 -05:00
23c27cffb3 Chore: Rename Id to ID in alerting models (#62777)
* Chore: Rename Id to ID in alerting models

* Add xorm tags for datasource

* Add xorm tag for uid
2023-02-02 17:22:43 +01:00
f8ec35e643 Chore: Fix goimports grouping (#62427)
* fix goimports order

* fix goimports order
2023-01-30 09:38:51 +01:00
e5920c211e Chore: Fix random indices for slices in test files (#61884)
* Fix random indices for slices in test files

* Empty commit
2023-01-24 15:07:37 -03:00
c0913ce718 SSE: Fix math expression to support NoData results (#61721)
* update perFloat to support NoData
* update union to correctly handle no-data
2023-01-20 09:53:03 -05:00
5e8866ed5a SSE: DSNode to parse only one response (#61644)
* do not iterate through responses
* log type of the response in all exit cases
2023-01-18 13:06:10 -05:00
ef6d73e575 SSE: Resample command to support NoData (#61708)
* add test for nil value
2023-01-18 12:39:38 -05:00
402dc5e4d6 Alerting: Redo refactoring from reverted fix in #56812 (#61051)
This pull request re-applies the refactoring of ConditionsCmd from a
reverted fix #56812 for mathexp.noData. It does not add the fix, or
tests for the fix, because those were added in #56816. We use the
additional test coverage added in #56816 and #58650 to avoid the
reoccurrence of regressions that caused us to revert #56812 the
first time.
2023-01-09 17:01:19 +00:00
6dbe3b555f Plugins: Refactor forward of cookies, OAuth token and header modifications by introducing client middlewares (#58132)
Adding support for backend plugin client middlewares. This allows headers in outgoing 
backend plugin and HTTP requests to be modified using client middlewares.

The following client middlewares added:
Forward cookies: Will forward incoming HTTP request Cookies to outgoing plugins.Client 
and HTTP requests if the datasource has enabled forwarding of cookies (keepCookies).
Forward OAuth token: Will set OAuth token headers on outgoing plugins.Client and HTTP 
requests if the datasource has enabled Forward OAuth Identity (oauthPassThru).
Clear auth headers: Will clear any outgoing HTTP headers that was part of the incoming 
HTTP request and used when authenticating to Grafana.
The current suggested way to register client middlewares is to have a separate package, 
pluginsintegration, responsible for bootstrap/instantiate the backend plugin client with 
middlewares and/or longer term bootstrap/instantiate plugin management. 

Fixes #54135
Related to #47734
Related to #57870
Related to #41623
Related to #57065
2022-12-01 19:08:36 +01:00
110fdf4da9 Alerting: Fix ConditionsCmd behavior when last is No Data (#58953) 2022-11-28 21:56:22 +00:00
5623b5afaf SSE: Make sure to forward headers, user and cookies/OAuth token (#58897)
Fixes #58793 and Fixes https://github.com/grafana/azure-data-explorer-datasource/issues/513

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-11-28 07:40:06 -05:00