99 Commits

Author SHA1 Message Date
b80fbe03f0 Plugins: Refactor Plugin Management (#40477)
* add core plugin flow

* add instrumentation

* move func

* remove cruft

* support external backend plugins

* refactor + clean up

* remove comments

* refactor loader

* simplify core plugin path arg

* cleanup loggers

* move signature validator to plugins package

* fix sig packaging

* cleanup plugin model

* remove unnecessary plugin field

* add start+stop for pm

* fix failures

* add decommissioned state

* export fields just to get things flowing

* fix comments

* set static routes

* make image loading idempotent

* merge with backend plugin manager

* re-use funcs

* reorder imports + remove unnecessary interface

* add some TODOs + remove unused func

* remove unused instrumentation func

* simplify client usage

* remove import alias

* re-use backendplugin.Plugin interface

* re order funcs

* improve var name

* fix log statements

* refactor data model

* add logic for dupe check during loading

* cleanup state setting

* refactor loader

* cleanup manager interface

* add rendering flow

* refactor loading + init

* add renderer support

* fix renderer plugin

* reformat imports

* track errors

* fix plugin signature inheritance

* name param in interface

* update func comment

* fix func arg name

* introduce class concept

* remove func

* fix external plugin check

* apply changes from pm-experiment

* fix core plugins

* fix imports

* rename interface

* comment API interface

* add support for testdata plugin

* enable alerting + use correct core plugin contracts

* slim manager API

* fix param name

* fix filter

* support static routes

* fix rendering

* tidy rendering

* get tests compiling

* fix install+uninstall

* start finder test

* add finder test coverage

* start loader tests

* add test for core plugins

* load core + bundled test

* add test for nested plugin loading

* add test files

* clean interface + fix registering some core plugins

* refactoring

* reformat and create sub packages

* simplify core plugin init

* fix ctx cancel scenario

* migrate initializer

* remove Init() funcs

* add test starter

* new logger

* flesh out initializer tests

* refactoring

* remove unused svc

* refactor rendering flow

* fixup loader tests

* add enabled helper func

* fix logger name

* fix data fetchers

* fix case where plugin dir doesn't exist

* improve coverage + move dupe checking to loader

* remove noisy debug logs

* register core plugins automagically

* add support for renderer in catalog

* make private func + fix req validation

* use interface

* re-add check for renderer in catalog

* tidy up from moving to auto reg core plugins

* core plugin registrar

* guards

* copy over core plugins for test infra

* all tests green

* renames

* propagate new interfaces

* kill old manager

* get compiling

* tidy up

* update naming

* refactor manager test + cleanup

* add more cases to finder test

* migrate validator to field

* more coverage

* refactor dupe checking

* add test for plugin class

* add coverage for initializer

* split out rendering

* move

* fixup tests

* fix uss test

* fix frontend settings

* fix grafanads test

* add check when checking sig errors

* fix enabled map

* fixup

* allow manual setup of CM

* rename to cloud-monitoring

* remove TODO

* add installer interface for testing

* loader interface returns

* tests passing

* refactor + add more coverage

* support 'stackdriver'

* fix frontend settings loading

* improve naming based on package name

* small tidy

* refactor test

* fix renderer start

* make cloud-monitoring plugin ID clearer

* add plugin update test

* add integration tests

* don't break all if sig can't be calculated

* add root URL check test

* add more signature verification tests

* update DTO name

* update enabled plugins comment

* update comments

* fix linter

* revert fe naming change

* fix errors endpoint

* reset error code field name

* re-order test to help verify

* assert -> require

* pm check

* add missing entry + re-order

* re-check

* dump icon log

* verify manager contents first

* reformat

* apply PR feedback

* apply style changes

* fix one vs all loading err

* improve log output

* only start when no signature error

* move log

* rework plugin update check

* fix test

* fix multi loading from cfg.PluginSettings

* improve log output #2

* add error abstraction to capture errors without registering a plugin

* add debug log

* add unsigned warning

* e2e test attempt

* fix logger

* set home path

* prevent panic

* alternate

* ugh.. fix home path

* return renderer even if not started

* make renderer plugin managed

* add fallback renderer icon, update renderer badge + prevent changes when renderer is installed

* fix icon loading

* rollback renderer changes

* use correct field

* remove unneccessary block

* remove newline

* remove unused func

* fix bundled plugins base + module fields

* remove unused field since refactor

* add authorizer abstraction

* loader only returns plugins expected to run

* fix multi log output
2021-11-01 10:53:33 +01:00
e4297006f6 Chore: Propagate context for data source provisioning (#40235)
* context all the things

* apply feedback

* rollback some alerting changes

* rollback some alerting changes #2

* more rollbacks

* more rollbacks #2

* more rollbacks #3

* more rollbacks #4

* fix integration test

* add missing context

* add missing and remove incorrect dispatch
2021-10-18 17:06:19 +02:00
57fcfd578d Chore: replace macaron with web package (#40136)
* replace macaron with web package

* add web.go
2021-10-11 14:30:59 +02:00
722c414fef Encryption: Refactor securejsondata.SecureJsonData to stop relying on global functions (#38865)
* Encryption: Add support to encrypt/decrypt sjd

* Add datasources.Service as a proxy to datasources db operations

* Encrypt ds.SecureJsonData before calling SQLStore

* Move ds cache code into ds service

* Fix tlsmanager tests

* Fix pluginproxy tests

* Remove some securejsondata.GetEncryptedJsonData usages

* Add pluginsettings.Service as a proxy for plugin settings db operations

* Add AlertNotificationService as a proxy for alert notification db operations

* Remove some securejsondata.GetEncryptedJsonData usages

* Remove more securejsondata.GetEncryptedJsonData usages

* Fix lint errors

* Minor fixes

* Remove encryption global functions usages from ngalert

* Fix lint errors

* Minor fixes

* Minor fixes

* Remove securejsondata.DecryptedValue usage

* Refactor the refactor

* Remove securejsondata.DecryptedValue usage

* Move securejsondata to migrations package

* Move securejsondata to migrations package

* Minor fix

* Fix integration test

* Fix integration tests

* Undo undesired changes

* Fix tests

* Add context.Context into encryption methods

* Fix tests

* Fix tests

* Fix tests

* Trigger CI

* Fix test

* Add names to params of encryption service interface

* Remove bus from CacheServiceImpl

* Add logging

* Add keys to logger

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

* Add missing key to logger

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

* Undo changes in markdown files

* Fix formatting

* Add context to secrets service

* Rename decryptSecureJsonData to decryptSecureJsonDataFn

* Name args in GetDecryptedValueFn

* Add template back to NewAlertmanagerNotifier

* Copy GetDecryptedValueFn to ngalert

* Add logging to pluginsettings

* Fix pluginsettings test

Co-authored-by: Tania B <yalyna.ts@gmail.com>
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2021-10-07 17:33:50 +03:00
6005e6572d Chore: fix param names in ctx.ParamsInt64 (#39223) 2021-09-15 12:14:15 +02:00
063160aae2 Chore: pass url parameters through context.Context (#38826)
* pass url parameters through context.Context

* fix url param names without colon prefix

* change context params to vars

* replace url vars in tests using new api

* rename vars to params

* add some comments

* rename seturlvars to seturlparams
2021-09-14 18:34:56 +02:00
78596a6756 Migrate to Wire for dependency injection (#32289)
Fixes #30144

Co-authored-by: dsotirakis <sotirakis.dim@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Leon Sorokin <leeoniya@gmail.com>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: spinillos <selenepinillos@gmail.com>
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
2021-08-25 15:11:22 +02:00
db2e0d46b7 Update http error code when datasource health check fails (#37895)
Since the datasource settings are configured by the user, the health check error could be a client error. A 506 error may be sufficient as well.
2021-08-16 09:49:34 +03:00
e799257637 Live: stream resubmit on ds change, fix old ds settings in RunStream (#34130) 2021-05-18 21:39:56 +03:00
5ec530f3fc API: Return 409 on datasource version conflict (#32425)
Signed-off-by: bergquist <carl.bergquist@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2021-03-29 16:03:15 +02:00
eb241b960f API: Datasource endpoint should return 400 bad request if id and orgId is invalid (#32392)
* return 400 bad request if id and orgId is invalid

Signed-off-by: bergquist <carl.bergquist@gmail.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-03-29 08:56:58 +02:00
87c3a2b790 PluginManager: Make Plugins, Renderer and DataSources non-global (#31866)
* PluginManager: Make Plugins and DataSources non-global

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix integration tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Replace outdated command

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* DashboardService: Ensure it gets constructed with necessary parameters

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix build

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* DashboardService: Ensure it gets constructed with necessary parameters

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Remove dead code

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix test

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix test

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Remove FocusConvey

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix test

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Remove dead code

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Undo interface changes

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Backend: Move tsdbifaces.RequestHandler to plugins.DataRequestHandler

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Rename to DataSourceCount

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Consolidate dashboard interfaces into one

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix test

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix dashboard integration tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-03-17 16:06:10 +01:00
b79e61656a Introduce TSDB service (#31520)
* Introduce TSDB service

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>
2021-03-08 07:02:49 +01:00
6db4b40d5b Data source list: Use Card component (#31326)
* Replace DataSourcesListItem with Card

* Add tests

* Remove unused styles

* Make card heading semi bold

* Make heading semi-bold

* Show type name instead of type id

* Fix key warning

* Update Card

* Fix tests

* Make typeName optional

* remove styling that was just a test

* Make typeName non-optional and fix tests

* Update list key

Co-authored-by: Torkel Ödegaard <torkel@grafana.com>
2021-02-22 14:02:10 +02:00
3d41267fc4 Chore: Moves common and response into separate packages (#30298)
* Chore: moves common and response into separate packages

* Chore: moves common and response into separate packages

* Update pkg/api/utils/common.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Chore: changes after PR comments

* Chore: move wrap to routing package

* Chore: move functions in common to response package

* Chore: move functions in common to response package

* Chore: formats imports

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-01-15 14:43:20 +01:00
1c1a800bc0 API: Add by UID routes for data sources (#29884)
- also add Get by UID+OrgID to datasource cache
- Refactor backend commands for Delete and Get to be unified
2021-01-13 13:16:27 -05:00
375e8e4fd0 SQLStore: customise the limit of retrieved datasources per organisation (#29358)
* SQLStore: customise the limit of retrieved datasources per organisation

* update all suggestions regarding nil or 0 as default

* Apply suggestions from code review

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

* correct default.ini description + adding unittest

* Apply suggestions from code review

Co-authored-by: Sofia Papagiannaki <papagian@users.noreply.github.com>

* modify unittest name

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
Co-authored-by: Sofia Papagiannaki <papagian@users.noreply.github.com>
2020-12-28 12:24:42 +01:00
294770f411 Chore: Handle wrapped errors (#29223)
* Chore: Handle wrapped errors

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2020-11-19 13:34:28 +01:00
8dc75f79f7 API: Enrich endpoints with ID (#26691) 2020-07-31 08:22:09 +02:00
41d432b5ae Chore: Enable whitespace linter (#25903)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-07-06 20:17:28 +02:00
d352c213b3 API: Recognize MSSQL data source URLs (#25629)
* API: Recognize MSSQL URLs

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Move MSSQL URL validation into mssql package

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-06-17 11:17:11 +02:00
c0f3b2929c Backend plugins: Refactor to allow shared contract between core and external backend plugins (#25472)
Refactor to allow shared contract between core and external backend plugins 
allowing core backend data sources in Grafana to be implemented in same 
way as an external backend plugin.
Use v0.67.0 of sdk.
Add tests for verifying plugin is restarted when process is killed.
Enable strict linting for backendplugin packages
2020-06-11 16:14:05 +02:00
164242f5fd Data sources: Don't fail if URL doesn't specify protocol (#24497) 2020-05-12 13:04:18 +02:00
80e1191ca0 Backend Plugins: (breaking change) Add PluginContext (#23788)
* breaking change for newer backend plugins
* use exported protobuf converters and sdk types to reduce duplicate code
* uses grafana-plugin-sdk-go@v0.54.0
2020-04-23 14:08:21 -04:00
7d88018531 DataSourceProxy: Handle URL parsing error (#23731)
* pluginproxy: Handle URL parsing error
* pkg/api: Validate data source URLs
* pkg/api: Return 400 for URL validation error
2020-04-22 10:30:06 +02:00
871ad73414 Backend plugins: Renderer v2 plugin (#23625)
grafana-plugin-model is legacy and is replaced by new backend 
plugins SDK and architecture. Renderer is not part of SDK and 
we want to keep it that way for now since it's highly unlikely there 
will be more than one kind of renderer plugin.
So this PR adds support for renderer plugin v2.
Also adds support sending a Device Scale Factor parameter to the 
plugin v2 remote rendering service and by that replaces #22474.
Adds support sending a Headers parameter to the plugin v2 and
remote rendering service which for now only include 
Accect-Language header (the user locale in browser when using 
Grafana), ref grafana/grafana-image-renderer#45.
Fixes health check json details response.
Adds image renderer plugin configuration settings in defaults.ini 
and sample.ini.

Co-Authored-By: Arve Knudsen <arve.knudsen@gmail.com>
2020-04-21 16:16:41 +02:00
e5dd7efdee Provisioning: Allows specifying uid for datasource and use that in derived fields (#23585)
* Add uid to datasource

* Fix uid passing when provisioning

* Better error handling and Uid column type change

* Fix test and strict null error counts

* Add backend tests

* Add tests

* Fix strict null checks

* Update test

* Improve tests

* Update pkg/services/sqlstore/datasource.go

Co-Authored-By: Arve Knudsen <arve.knudsen@gmail.com>

* Variable rename

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-04-20 15:48:38 +02:00
579abad9cc Plugins: Return jsondetails as an json object instead of raw json on datasource healthchecks. (#22859) 2020-03-18 12:08:52 +01:00
ebc9549cbc Add support for sending health check to datasource plugins. (#22771)
closes #21519
ref grafana/grafana-plugin-sdk-go#93
2020-03-13 12:31:44 +01:00
5f94d31da9 Send jsondata for Datasources on DatasourceConfig for backend plugins (#22681)
ref https://github.com/grafana/grafana-plugin-sdk-go/pull/84
2020-03-10 15:18:27 +01:00
3fdd2648b1 Chore: Avoid aliasing importing models in api package (#22492) 2020-03-04 12:57:20 +01:00
e6cec8dbdc Backend plugins: Prepare and clean request headers before resource calls (#22321)
Moves common request proxy utilities to proxyutil package with
support for removing X-Forwarded-Host, X-Forwarded-Port,
X-Forwarded-Proto headers, setting X-Forwarded-For header
and cleaning Cookie header.
Using the proxyutil package to prepare and clean request
headers before resource calls.

Closes #21512
2020-03-03 11:45:16 +01:00
9d7c74ef91 Backend Plugins: Provide proper plugin config to plugins (#21985)
Properly provides plugin configs to backend plugins.
Uses v0.16.0 of grafana-plugin-sdk-go-

Ref #21512
Ref #19667
2020-02-19 19:17:05 +01:00
0390b5601e Backend plugins: Implement support for resources (#21805)
Implements initial support for resources using v0.14.0 of SDK.

Ref #21512
2020-01-31 11:15:50 +01:00
b5f0a5d5ca Api: Readonly datasources should not be created via the API (#19006)
* Readonly datasources should not be created via the API

* Return correct `ReadOnly` value in GetDataSourceByName
2019-09-17 12:29:43 +03:00
3245227016 permissions: cleanup. 2018-10-17 15:58:52 +02:00
1329c70965 codestyle 2018-10-16 14:22:50 +02:00
ec9ed5c830 Removes unused code. 2018-10-11 13:52:23 +02:00
ccfcf02e33 Refactors ds permissions to a filter. 2018-10-10 14:25:57 +02:00
162a95bff9 wip: going in circles 2018-10-01 19:31:03 +02:00
b3c78f1265 wip: data source permissions hooks 2018-10-01 15:38:55 +02:00
eb1b9405b2 return proper payload from api when updating datasource 2018-08-07 17:56:02 +02:00
defb0396b7 Return a 404 when deleting a datasource through the API if it doesn't exist and add a test for it to confirm #12313 2018-06-22 21:15:36 -05:00
0a415c50d0 Make golint happier 2018-03-22 22:38:44 +01:00
7aab6a8887 Make golint happier 2018-03-22 12:40:21 +01:00
c0ecdee375 rename Context to ReqContext 2018-03-07 11:54:50 -05:00
338655dd37 move Context and session out of middleware 2018-03-06 18:16:49 -05:00
909601d6ab api: fix so that datasources functions returns Response 2017-11-16 16:29:05 +01:00
36676e23c1 datasource as cfg: adds readonly datasources 2017-11-15 14:30:32 +01:00
d68bfaa8be datasource as cfg: typo 2017-10-25 12:21:06 +02:00