58 Commits

Author SHA1 Message Date
286b9e08e9 Plugins: Bump Plugin SDK version and address instance management breaking changes (#68900)
* bump sdk and propagate ctx

* fix tests
2023-05-24 10:19:34 +02:00
531caec602 Loki: Add feature flag to enable dataplane-compliant metric frames (#66017)
* loki: add feature flag to enable dataplane-compatible metric frames

* linter fix
2023-04-13 15:07:08 +02:00
7261c6f7cd DataSourceWithBackend: Add queryGroupId to find correlated/related queries (#64587)
* add `correlationId` to queries

* trace correlation id in backend

* add correlation id to loki's span

* add correlation id to query chunks

* fix test

* fix DataSourceWithBackend test

* change to `queryGroupId`

* remove empty line

* fix test in `DataSourceWithBackend`
2023-03-17 15:48:13 +01:00
473013e3f5 Loki: Add error handling to CallResource (#64439)
* surface loki error message for `CallResource`

* use `data.Message` instead or `errorMessage`

* change struct coming from Loki

* remove whitespace
2023-03-09 11:12:33 +01:00
48a374f50b loki: implement schema for loki query (#62114)
* loki: implement schema for loki query

* removed forgotten code

* better types

* adjust cue-maturity-level

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* sync docs to schema change

* removed forgotten go code

* updated schema docs

Co-authored-by: J Stickler <julie.stickler@grafana.com>

* updated schema docs

Co-authored-by: J Stickler <julie.stickler@grafana.com>

* update schema docs

Co-authored-by: J Stickler <julie.stickler@grafana.com>

* update schema docs

Co-authored-by: J Stickler <julie.stickler@grafana.com>

* updated schema docs

Co-authored-by: J Stickler <julie.stickler@grafana.com>

* updated schema docs

* add documentation to the resolution field

* move direction-field to schema

* loki: cue: moved deprecated fields to the bottom

---------

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
Co-authored-by: J Stickler <julie.stickler@grafana.com>
2023-02-02 16:18:30 +00:00
b2c8126e6e Loki: Show query size approximation (#62109)
* feat: make api request to /loki/api/v1/index/stats

* fix: add /index/stats to callResource valid urls

* feat: make call to getQueryStats when the query changes

* feat: render user tooltip displaying the estimated value for processed data

* fix: add new props to component tests

* test: add tests for query size estimation

* fix: disable error message on request failure

* refactor: add suggestions from code review

* refactor: only pass required query string
2023-01-27 17:32:53 +00:00
d0e95f8c95 Loki: Add X-Query-Tags header for logs sample and data sample (#62333) 2023-01-27 16:41:40 +01:00
c35c689a96 Plugins: Automatically forward plugin request HTTP headers in outgoing HTTP requests (#60417)
Automatically forward core plugin request HTTP headers in outgoing HTTP requests. 
Core datasource plugin authors don't have to specifically handle forwarding of HTTP 
headers, e.g. do not have to "hardcode" the header-names in the datasource plugin, 
if not having custom needs.

Fixes #57065
2022-12-21 13:25:58 +01:00
f1ef63791a Fix #58598 X-ID-Token header missing on Loki Datasource (#58784)
* Fix #58598 X-ID-Token header missing on Loki Datasource

* Remove unecessary continue statements

* Add getAuthHeadersForCallResource unit tests

* Fix test and switch statement issues introduced during merge
2022-11-23 11:31:50 +01:00
08e87a217c Loki: Add gzip compression to resource calls (#59059)
* Loki: Add compression to `callResource`

* add missing tests

* fix formatting
2022-11-22 14:02:30 +01:00
bc280d0749 Datasource Loki: preserve header X-ID-Token (#57878) 2022-11-07 16:53:42 +01:00
fa043efb2c Fix: correctly close trace span in Prometheus and Loki data sources (#57774)
* rename span for loki
2022-10-28 10:17:56 -04:00
facf2b1ee8 Chore: Update prometheus, loki, graphite and influx plugins to support contextual logs. (#57708) 2022-10-27 12:05:06 -04:00
145b970557 Propagate all headers on main QueryData (#56896) 2022-10-13 14:18:53 -05:00
8fd9cb4854 loki: add feature-toggle check to live-mode (#49012) 2022-06-13 08:33:46 +02:00
46d6573968 loki: add cookie-handling functionality (#49978) 2022-06-02 11:52:27 +02:00
da74dba7c8 Loki: backend: use streaming JSON parser, try2 (#48752)
* converter: remove __name__ customization because Loki does not do that

Loki does not handle __name__ in a special way.
for Prometheus, the caller can implement the formatting
by themselves

* converter: change labels-formatting

the labels.String() method does not handle strange values well

* loki: backend: use streaming-json parser

* more idiomatic code

Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>

* simpler row-length check

* simpler code

* fixed converter/prom tests

Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
2022-05-05 13:09:01 +02:00
02aa1cd1c5 loki backend mode forward-oauth (#48401)
* loki: backend: add forward oauth credentials functionality

* removed obsolete comment
2022-05-05 12:42:50 +02:00
3cade2f669 Revert "Loki: backend: use streaming JSON parser (#47656)" (#48747)
This reverts commit 46b40b6e82417661078be5f1c4a9d802e440d9ec.
2022-05-05 12:02:53 +02:00
46b40b6e82 Loki: backend: use streaming JSON parser (#47656)
* converter: remove __name__ customization because Loki does not do that

Loki does not handle __name__ in a special way.
for Prometheus, the caller can implement the formatting
by themselves

* converter: change labels-formatting

the labels.String() method does not handle strange values well

* loki: backend: use streaming-json parser

* more idiomatic code

Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>

* simpler row-length check

* simpler code

Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
2022-05-05 08:47:40 +02:00
ae926c2144 loki: better resource-call urls (#48250) 2022-04-28 14:28:57 +02:00
c73b9d1818 Loki: do not use the old labels-api (#48185)
* loki: do not use old labels-api

* adjust allowed-url-list

* fixed tests
2022-04-26 11:51:26 +02:00
d8a754c4a0 loki: send metadata requests through backend (#48063) 2022-04-25 13:16:14 +02:00
79c06fdddc Loki: add backend-forward mode to queries, update log-row-context (#47726)
* loki: add helper function to sort dataframe by time

* loki: add direction-attribute to queries

* loki: make log-row-context code backward-compatible

* better comment

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>

* fixed test

* simplified code

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>
2022-04-20 13:52:15 +02:00
796bc27f75 Loki: support live streaming (#42804) 2022-03-01 14:46:52 -08:00
1cad35ea67 loki: backend mode: support all query types (#45619)
* loki: backend mode: support all query types

* loki: backend: adjust vector-parsing field-names

* loki: backend: no interval for streams-dataframes

* loki: backend: enable more query types

* better variable name

* removed unnecessary code

* improve frame-processing

* more unit tests

* improved code

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>

* remove unused code

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>

* simplify code

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>

* lint fix

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>
2022-02-25 09:14:17 +01:00
5d704fd46e Loki: backend: use custom http code (#44643)
* Loki: backend: use custom http code

* simplfied comment

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>
2022-02-18 14:35:39 +01:00
a757a57c6e Prometheus: Show executed query in inspector (#45362)
* Prometheus: Show executed query in inspector

* Loki: Improving test
2022-02-15 15:02:03 +01:00
96ae44ee2c loki: alerting: do not round timestamps to seconds (#45057) 2022-02-10 12:53:46 +01:00
d9d1f8520e loki: backend-queries: set interval-value in dataframe (#44965)
* loki: backend-queries: set interval-value in dataframe

* fixed unit test

* added interval-specific unit test
2022-02-07 13:02:04 +01:00
3873c5e7b5 loki: alerting: better error-handling (#44700) 2022-02-01 13:34:03 +01:00
b42161a713 Loki: add from JSON to dataframes tests (#44375)
* Loki: add from JSON to dataframes tests

* fixed wrong content-type

* added -Inf test-case
2022-01-27 12:43:44 +01:00
7fbc7d019a Plugins: Core plugins register via backend factory provider (#43171)
* refactoring store interface and init flow

* fix import

* fix linter

* refactor resource calling

* load with class

* re-order args

* fix tests

* fix linter

* remove old creator

* add custom config struct

* fix some tests

* cleanup

* fix

* tackle plugins

* fix linter

* refactor and fix test

* add connect failure error

* add fix for azure, cloud monitoring and test data

* restructure

* remove unused err

* add fake tracer for test

* fix grafana ds plugin
2022-01-20 18:16:22 +01:00
30aa24a183 Chore: Implement OpenTelemtry in Grafana (#42674)
* Separate Tracer interface to TracerService and Tracer

* Fix lint

* Fix:Make it possible to start spans for both opentracing and opentelemetry in ds proxy

* Add span methods, use span interface for rest of tracing

* Fix logs in tracing

* Fix tests that are related to tracing

* Fix resourcepermissions test

* Fix some tests

* Fix more tests

* Add TracingService to wire cli runner

* Remove GlobalTracer from bus

* Renaming test function

* Remove GlobalTracer from TSDB

* Replace GlobalTracer in api

* Adjust tests to the InitializeForTests func

* Remove GlobalTracer from services

* Remove GlobalTracer

* Remove bus.NewTest

* Remove Tracer interface

* Add InitializeForBus

* Simplify tests

* Clean up tests

* Rename TracerService to Tracer

* Update pkg/middleware/request_tracing.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Initialize tracer before passing it to SQLStore initialization in commands

* Remove tests for opentracing

* Set span attributes correctly, remove unnecessary trace initiliazation form test

* Add tracer instance to newSQLStore

* Fix changes due to rebase

* Add modified tracing middleware test

* Fix opentracing implementation tags

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-01-20 11:10:12 +01:00
546818819b Loki: use queryType rather than range|instant flags (#43727) 2022-01-07 10:00:11 -08:00
e4ba5f17dd add core plugins thru store (#43085) 2021-12-14 15:22:40 +01:00
3232ae08c0 loki: removed unused variable (#42909) 2021-12-09 13:16:52 +01:00
6686f8c345 loki: rename ResponseModel to QueryModel (#42794) 2021-12-07 07:57:52 +01:00
24d4c8a9d1 Loki: alerting: handle variables like __interval and __range (#42126) 2021-11-24 16:56:53 +01:00
fe84454bb0 loki: alerting: simplified config (#42147)
* loki: alerting: simplified config

* adjusted config handling
2021-11-24 11:52:33 +01:00
037fd9cc8c Loki: alerting: adjust step-calculation to be the same as in frontend (#42033)
* loki: alerting: adjust step-calculation to be the same as in frontend

* loki: simplified test

* lint fix
2021-11-22 14:28:51 +01:00
2fbc6b1d2f loki: simplify interval calculation (#41857) 2021-11-19 17:20:32 +01:00
b80fbe03f0 Plugins: Refactor Plugin Management (#40477)
* add core plugin flow

* add instrumentation

* move func

* remove cruft

* support external backend plugins

* refactor + clean up

* remove comments

* refactor loader

* simplify core plugin path arg

* cleanup loggers

* move signature validator to plugins package

* fix sig packaging

* cleanup plugin model

* remove unnecessary plugin field

* add start+stop for pm

* fix failures

* add decommissioned state

* export fields just to get things flowing

* fix comments

* set static routes

* make image loading idempotent

* merge with backend plugin manager

* re-use funcs

* reorder imports + remove unnecessary interface

* add some TODOs + remove unused func

* remove unused instrumentation func

* simplify client usage

* remove import alias

* re-use backendplugin.Plugin interface

* re order funcs

* improve var name

* fix log statements

* refactor data model

* add logic for dupe check during loading

* cleanup state setting

* refactor loader

* cleanup manager interface

* add rendering flow

* refactor loading + init

* add renderer support

* fix renderer plugin

* reformat imports

* track errors

* fix plugin signature inheritance

* name param in interface

* update func comment

* fix func arg name

* introduce class concept

* remove func

* fix external plugin check

* apply changes from pm-experiment

* fix core plugins

* fix imports

* rename interface

* comment API interface

* add support for testdata plugin

* enable alerting + use correct core plugin contracts

* slim manager API

* fix param name

* fix filter

* support static routes

* fix rendering

* tidy rendering

* get tests compiling

* fix install+uninstall

* start finder test

* add finder test coverage

* start loader tests

* add test for core plugins

* load core + bundled test

* add test for nested plugin loading

* add test files

* clean interface + fix registering some core plugins

* refactoring

* reformat and create sub packages

* simplify core plugin init

* fix ctx cancel scenario

* migrate initializer

* remove Init() funcs

* add test starter

* new logger

* flesh out initializer tests

* refactoring

* remove unused svc

* refactor rendering flow

* fixup loader tests

* add enabled helper func

* fix logger name

* fix data fetchers

* fix case where plugin dir doesn't exist

* improve coverage + move dupe checking to loader

* remove noisy debug logs

* register core plugins automagically

* add support for renderer in catalog

* make private func + fix req validation

* use interface

* re-add check for renderer in catalog

* tidy up from moving to auto reg core plugins

* core plugin registrar

* guards

* copy over core plugins for test infra

* all tests green

* renames

* propagate new interfaces

* kill old manager

* get compiling

* tidy up

* update naming

* refactor manager test + cleanup

* add more cases to finder test

* migrate validator to field

* more coverage

* refactor dupe checking

* add test for plugin class

* add coverage for initializer

* split out rendering

* move

* fixup tests

* fix uss test

* fix frontend settings

* fix grafanads test

* add check when checking sig errors

* fix enabled map

* fixup

* allow manual setup of CM

* rename to cloud-monitoring

* remove TODO

* add installer interface for testing

* loader interface returns

* tests passing

* refactor + add more coverage

* support 'stackdriver'

* fix frontend settings loading

* improve naming based on package name

* small tidy

* refactor test

* fix renderer start

* make cloud-monitoring plugin ID clearer

* add plugin update test

* add integration tests

* don't break all if sig can't be calculated

* add root URL check test

* add more signature verification tests

* update DTO name

* update enabled plugins comment

* update comments

* fix linter

* revert fe naming change

* fix errors endpoint

* reset error code field name

* re-order test to help verify

* assert -> require

* pm check

* add missing entry + re-order

* re-check

* dump icon log

* verify manager contents first

* reformat

* apply PR feedback

* apply style changes

* fix one vs all loading err

* improve log output

* only start when no signature error

* move log

* rework plugin update check

* fix test

* fix multi loading from cfg.PluginSettings

* improve log output #2

* add error abstraction to capture errors without registering a plugin

* add debug log

* add unsigned warning

* e2e test attempt

* fix logger

* set home path

* prevent panic

* alternate

* ugh.. fix home path

* return renderer even if not started

* make renderer plugin managed

* add fallback renderer icon, update renderer badge + prevent changes when renderer is installed

* fix icon loading

* rollback renderer changes

* use correct field

* remove unneccessary block

* remove newline

* remove unused func

* fix bundled plugins base + module fields

* remove unused field since refactor

* add authorizer abstraction

* loader only returns plugins expected to run

* fix multi log output
2021-11-01 10:53:33 +01:00
f172701043 IntervalV2: Use maxDataPoints to calculate correct interval (#39036)
* Use max data points to calculate interval

* Fix mockCalculator to mach Calculator

* Fix incorrect merge, replace queryModel with query
2021-09-13 14:58:32 +02:00
dc36f15fbb Prometheus: Revert stepMode functionality (#38982)
* Revert "Prometheus: add functionality to specify desired step interval in dashboards panels (#36422)"

This reverts commit ddf5b65c51a36168e803d471f65c9f9e81abbf57.
Co-authored-by: Ivana Huckova <ivana.huckova@gmail.com>

* Revert "Explore: add functionality for supporting different step modes in prometheus (#37829)"

This reverts commit f433cfd8d9978ee37b0ceeb0b679b335a0acf193.
Co-authored-by: Ivana Huckova <ivana.huckova@gmail.com>

* Revert stepMode BE implementation from #36796
Co-authored-by: "Ivana Huckova" <ivana.huckova@gmail.com>
2021-09-09 14:05:08 +02:00
c989f0fbbe SQL: Migrate to use SDK contracts (#36635)
* convert SQLs to use sdk contracts

* make draft

* postgres

* intermedia

* get datasourceinfo filled at the beginning of the service

* move the interval into package because of cyclict  import and fix all postgres tests

* fix mysql test

* fix mssql

* fix the test for pr https://github.com/grafana/grafana/issues/35839

* fix some issue about intervalv2 package

* update sql test

* wire migration for SQLs

* add sqls to the background process

* make it register instead of register and start

* revert formatting

* fix tests

* fix linter

* remove integration test

* Postgres test fix

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-09-07 09:35:37 +02:00
78596a6756 Migrate to Wire for dependency injection (#32289)
Fixes #30144

Co-authored-by: dsotirakis <sotirakis.dim@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Leon Sorokin <leeoniya@gmail.com>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: spinillos <selenepinillos@gmail.com>
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
2021-08-25 15:11:22 +02:00
1618d5ad6f use most appropriate func for registering core plugins (#38497) 2021-08-25 10:08:48 +02:00
d26f3cdd03 Loki: add support for resolution (#36710)
* Add input to specify min step

* Add stepInterval to as input to component

* Add onBlur to Input component

* Loki: add functionality for min step

* Loki: change name on props to step to make it more clear

* Loki: add resolution as a query option

* Loki: Add min,max,exact as step options

* Loki: add functionality for different step modes

* Loki: fix bug where step function isn't working

* Loki: fix bug where exact step isn't working

* Loki: change width of step input field

* Loki: add tests for adjustInterval function

* Loki: add check for max step oprio to make sure it's not below the safe interval

* Loki: fix bug with some tests

* Loki: fix bug with tests

* Explore: add tooltip to loki step function

* Loki: remove resolution as a logs option

* Loki: update snapshots

* Fix failing tests

* Loki: add select component for choosing resolution

* Loki: add functionality for calculating correct interval with resolution applied

* Loki: remove functionality for step mode

* Loki: remove tests for step mode

* Loki: add tooltip to line limit and resolution

* Loki: add backend support for resolution

* Loki: fixed backend bug where resolution was undefined

* Loki: add check for resolution
2021-08-16 14:02:13 +02:00
0563bc68fc Chore: Refactor loki to use SDK contracts (#37400)
* Refactor loki to use SDK contracts

* Register with service name

* Initialize interval calculator

* Return always created result

* Use go library instead of simplejson

* Update pkg/tsdb/loki/loki.go

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>

* Remove newline

* Merge with main

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>
2021-08-05 10:25:31 +02:00