43 Commits

Author SHA1 Message Date
4bc582570e Instrumentation: Proxy status code correction and various improvements (#47473)
For a proxied request, e.g. Grafana's datasource or plugin proxy:
If the request is cancelled, e.g. from the browser, the HTTP status code is 
now 499 Client closed request instead of 502 Bad gateway.
If the request times out, e.g. takes longer time than allowed, the HTTP status 
code is now 504 Gateway timeout instead of 502 Bad gateway.
This also means that request metrics and logs will get their status codes 
adjusted according to above.

Fixes #46337
Fixes #46338
2022-04-11 13:17:08 +02:00
6a8cbd8663 Plugins: Plugin settings refactor (#46246)
* remove bus and direct use of sqlStore

* add decryption to interface

* return nil

* rename field

* re-order fields

* rename file
2022-03-04 17:09:50 +01:00
9d654bb6b8 Chore: Remove bus from plugin proxy api (#44899)
* remove bus from plugin proxy api

* fixing the tests
2022-02-04 17:35:00 +01:00
56c3875bb9 Chore: Remove context.TODO (#43458)
* Remove context.TODO() from services

* Fix live test
2021-12-28 10:26:18 +01:00
2e3e7a7e55 Plugins: Plugin Store API returns DTO model (#41340)
* toying around

* fix refs

* remove unused fields

* go further

* add context

* ensure streaming handler is set
2021-11-17 12:04:22 +01:00
54de1078c8 remove the global log error/warn etc functions (#41404)
* remove the global log error/warn etc functions and use request context logger whenever possible
2021-11-08 17:56:56 +01:00
5652bde447 Encryption: Use secrets service (#40251)
* Use secrets service in pluginproxy

* Use secrets service in pluginxontext

* Use secrets service in pluginsettings

* Use secrets service in provisioning

* Use secrets service in authinfoservice

* Use secrets service in api

* Use secrets service in sqlstore

* Use secrets service in dashboardshapshots

* Use secrets service in tsdb

* Use secrets service in datasources

* Use secrets service in alerting

* Use secrets service in ngalert

* Break cyclic dependancy

* Refactor service

* Break cyclic dependancy

* Add FakeSecretsStore

* Setup Secrets Service in sqlstore

* Fix

* Continue secrets service refactoring

* Fix cyclic dependancy in sqlstore tests

* Fix secrets service references

* Fix linter errors

* Add fake secrets service for tests

* Refactor SetupTestSecretsService

* Update setting up secret service in tests

* Fix missing secrets service in multiorg_alertmanager_test

* Use fake db in tests and sort imports

* Use fake db in datasources tests

* Fix more tests

* Fix linter issues

* Attempt to fix plugin proxy tests

* Pass secrets service to getPluginProxiedRequest in pluginproxy tests

* Fix pluginproxy tests

* Revert using secrets service in alerting and provisioning

* Update decryptFn in alerting migration

* Rename defaultProvider to currentProvider

* Use fake secrets service in alert channels tests

* Refactor secrets service test helper

* Update setting up secrets service in tests

* Revert alerting changes in api

* Add comments

* Remove secrets service from background services

* Convert global encryption functions into vars

* Revert "Convert global encryption functions into vars"

This reverts commit 498eb19859eba364a2400a6d7e73236b1c9a5b37.

* Add feature toggle for envelope encryption

* Rename toggle

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
Co-authored-by: Joan López de la Franca Beltran <joanjan14@gmail.com>
2021-11-04 18:47:21 +02:00
b80fbe03f0 Plugins: Refactor Plugin Management (#40477)
* add core plugin flow

* add instrumentation

* move func

* remove cruft

* support external backend plugins

* refactor + clean up

* remove comments

* refactor loader

* simplify core plugin path arg

* cleanup loggers

* move signature validator to plugins package

* fix sig packaging

* cleanup plugin model

* remove unnecessary plugin field

* add start+stop for pm

* fix failures

* add decommissioned state

* export fields just to get things flowing

* fix comments

* set static routes

* make image loading idempotent

* merge with backend plugin manager

* re-use funcs

* reorder imports + remove unnecessary interface

* add some TODOs + remove unused func

* remove unused instrumentation func

* simplify client usage

* remove import alias

* re-use backendplugin.Plugin interface

* re order funcs

* improve var name

* fix log statements

* refactor data model

* add logic for dupe check during loading

* cleanup state setting

* refactor loader

* cleanup manager interface

* add rendering flow

* refactor loading + init

* add renderer support

* fix renderer plugin

* reformat imports

* track errors

* fix plugin signature inheritance

* name param in interface

* update func comment

* fix func arg name

* introduce class concept

* remove func

* fix external plugin check

* apply changes from pm-experiment

* fix core plugins

* fix imports

* rename interface

* comment API interface

* add support for testdata plugin

* enable alerting + use correct core plugin contracts

* slim manager API

* fix param name

* fix filter

* support static routes

* fix rendering

* tidy rendering

* get tests compiling

* fix install+uninstall

* start finder test

* add finder test coverage

* start loader tests

* add test for core plugins

* load core + bundled test

* add test for nested plugin loading

* add test files

* clean interface + fix registering some core plugins

* refactoring

* reformat and create sub packages

* simplify core plugin init

* fix ctx cancel scenario

* migrate initializer

* remove Init() funcs

* add test starter

* new logger

* flesh out initializer tests

* refactoring

* remove unused svc

* refactor rendering flow

* fixup loader tests

* add enabled helper func

* fix logger name

* fix data fetchers

* fix case where plugin dir doesn't exist

* improve coverage + move dupe checking to loader

* remove noisy debug logs

* register core plugins automagically

* add support for renderer in catalog

* make private func + fix req validation

* use interface

* re-add check for renderer in catalog

* tidy up from moving to auto reg core plugins

* core plugin registrar

* guards

* copy over core plugins for test infra

* all tests green

* renames

* propagate new interfaces

* kill old manager

* get compiling

* tidy up

* update naming

* refactor manager test + cleanup

* add more cases to finder test

* migrate validator to field

* more coverage

* refactor dupe checking

* add test for plugin class

* add coverage for initializer

* split out rendering

* move

* fixup tests

* fix uss test

* fix frontend settings

* fix grafanads test

* add check when checking sig errors

* fix enabled map

* fixup

* allow manual setup of CM

* rename to cloud-monitoring

* remove TODO

* add installer interface for testing

* loader interface returns

* tests passing

* refactor + add more coverage

* support 'stackdriver'

* fix frontend settings loading

* improve naming based on package name

* small tidy

* refactor test

* fix renderer start

* make cloud-monitoring plugin ID clearer

* add plugin update test

* add integration tests

* don't break all if sig can't be calculated

* add root URL check test

* add more signature verification tests

* update DTO name

* update enabled plugins comment

* update comments

* fix linter

* revert fe naming change

* fix errors endpoint

* reset error code field name

* re-order test to help verify

* assert -> require

* pm check

* add missing entry + re-order

* re-check

* dump icon log

* verify manager contents first

* reformat

* apply PR feedback

* apply style changes

* fix one vs all loading err

* improve log output

* only start when no signature error

* move log

* rework plugin update check

* fix test

* fix multi loading from cfg.PluginSettings

* improve log output #2

* add error abstraction to capture errors without registering a plugin

* add debug log

* add unsigned warning

* e2e test attempt

* fix logger

* set home path

* prevent panic

* alternate

* ugh.. fix home path

* return renderer even if not started

* make renderer plugin managed

* add fallback renderer icon, update renderer badge + prevent changes when renderer is installed

* fix icon loading

* rollback renderer changes

* use correct field

* remove unneccessary block

* remove newline

* remove unused func

* fix bundled plugins base + module fields

* remove unused field since refactor

* add authorizer abstraction

* loader only returns plugins expected to run

* fix multi log output
2021-11-01 10:53:33 +01:00
681218275e remove crit and trace (#40320) 2021-10-26 17:36:24 +02:00
57fcfd578d Chore: replace macaron with web package (#40136)
* replace macaron with web package

* add web.go
2021-10-11 14:30:59 +02:00
722c414fef Encryption: Refactor securejsondata.SecureJsonData to stop relying on global functions (#38865)
* Encryption: Add support to encrypt/decrypt sjd

* Add datasources.Service as a proxy to datasources db operations

* Encrypt ds.SecureJsonData before calling SQLStore

* Move ds cache code into ds service

* Fix tlsmanager tests

* Fix pluginproxy tests

* Remove some securejsondata.GetEncryptedJsonData usages

* Add pluginsettings.Service as a proxy for plugin settings db operations

* Add AlertNotificationService as a proxy for alert notification db operations

* Remove some securejsondata.GetEncryptedJsonData usages

* Remove more securejsondata.GetEncryptedJsonData usages

* Fix lint errors

* Minor fixes

* Remove encryption global functions usages from ngalert

* Fix lint errors

* Minor fixes

* Minor fixes

* Remove securejsondata.DecryptedValue usage

* Refactor the refactor

* Remove securejsondata.DecryptedValue usage

* Move securejsondata to migrations package

* Move securejsondata to migrations package

* Minor fix

* Fix integration test

* Fix integration tests

* Undo undesired changes

* Fix tests

* Add context.Context into encryption methods

* Fix tests

* Fix tests

* Fix tests

* Trigger CI

* Fix test

* Add names to params of encryption service interface

* Remove bus from CacheServiceImpl

* Add logging

* Add keys to logger

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

* Add missing key to logger

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

* Undo changes in markdown files

* Fix formatting

* Add context to secrets service

* Rename decryptSecureJsonData to decryptSecureJsonDataFn

* Name args in GetDecryptedValueFn

* Add template back to NewAlertmanagerNotifier

* Copy GetDecryptedValueFn to ngalert

* Add logging to pluginsettings

* Fix pluginsettings test

Co-authored-by: Tania B <yalyna.ts@gmail.com>
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2021-10-07 17:33:50 +03:00
063160aae2 Chore: pass url parameters through context.Context (#38826)
* pass url parameters through context.Context

* fix url param names without colon prefix

* change context params to vars

* replace url vars in tests using new api

* rename vars to params

* add some comments

* rename seturlvars to seturlparams
2021-09-14 18:34:56 +02:00
abfdfcf405 Plugins: Support multiple HTTP methods in plugin.json routes (#39069)
* Fix: support comma-separated methods in plugin.json

* update docs about multiple methods per route
2021-09-13 13:32:11 +02:00
c3ab2fdeb7 Macaron: remove custom Request type (#37874)
* remove macaron.Request, use http.Request instead

* remove com dependency from bindings module

* fix another c.Req.Request
2021-09-01 11:18:30 +02:00
d15cbe4b4e Macaron cleanup (#37795)
* simplify some dependency injection in macaron
* remove unused internal server error handler from macaron
* remove internal server error handler from the router
* remove unused combo router api
* remove unused parts of the macaron router
2021-08-30 11:48:34 +02:00
a2eda798e7 PluginManager: Make remaining plugin state non-global (#32094)
* PluginDashboards: Use plugin manager interface

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* PluginManager: Make panels non-global

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* PluginManager: Make apps non-global

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* PluginManager: Make static routes non-global

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* PluginManager: Make pluginTypes non-global

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-03-18 13:53:01 +01:00
b79e61656a Introduce TSDB service (#31520)
* Introduce TSDB service

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>
2021-03-08 07:02:49 +01:00
967e9b39e8 Fix panic when using complex dynamic URLs in app plugin routes (#27977)
* remove unused function to interpolate URLs

* share function to add headers between ds/plugin proxies

* stop performing unnecessary plugin setting lookup

* fix bug causing runtime errors when using complex templated URLs

* lower case util functions not used outside of pluginproxy package

* change test URL to a (valid) dummy URL to make intent clearer

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-11-17 10:56:42 +01:00
fbf0d2c086 Plugin proxy: Handle URL parsing errors (#29093)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-11-13 19:38:09 +01:00
4c56eb3991 Chore: Enable goprintffuncname and nakedret linters (#26376)
* Chore: Enable goprintffuncname linter
* Chore: Enable nakedret linter

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-07-23 08:14:39 +02:00
3fdd2648b1 Chore: Avoid aliasing importing models in api package (#22492) 2020-03-04 12:57:20 +01:00
2fff8f77dc move log package to /infra (#17023)
ref #14679

Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-05-13 08:45:54 +02:00
a86b8c8a04 http: remove dualstack since its deprecated (#16940)
Deprecated: Fast Fallback is enabled by default. To
disable, set FallbackDelay to a negative value.

https://golang.org/src/net/dial.go
2019-05-08 10:37:48 +02:00
3c21a121eb Plugins: Unifying alpha state & options for all plugins (#16530)
* app pages

* app pages

* workign example

* started alpha support

* remove app stuff

* show warning on alpha/beta panels

* put app back on plugin file

* fix go

* add enum for PluginType and PluginIncludeType

* Refactoring and moving settings to plugins section

fixes #16529
2019-04-12 13:46:42 +02:00
bbdc1c0e64 Add custom header with grafana user and a config switch for it 2019-03-14 16:33:19 +01:00
8261613b51 pkg/util/{ip.go,url.go}: Fix some golint issues
See,
$ gometalinter --vendor --deadline 10m --disable-all --enable=golint  ./...
ip.go:8:6⚠️ func SplitIpPort should be SplitIPPort (golint)
url.go:14:6⚠️ func NewUrlQueryReader should be NewURLQueryReader (golint)
url.go:9:6⚠️ type UrlQueryReader should be URLQueryReader (golint)
url.go:37:6⚠️ func JoinUrlFragments should be JoinURLFragments (golint)
2019-01-28 22:29:20 +01:00
10e86eda69 fix: #12461 introduced issues with route registration ordering, adding plugin static routes before plugins package had been initiated (#12474) 2018-07-01 23:35:50 -07:00
7aab6a8887 Make golint happier 2018-03-22 12:40:21 +01:00
c0ecdee375 rename Context to ReqContext 2018-03-07 11:54:50 -05:00
338655dd37 move Context and session out of middleware 2018-03-06 18:16:49 -05:00
9d53653647 Merge pull request #9378 from mattbostock/verify_tls
Bugfix: Always verify TLS unless explicitly told otherwise
2017-10-12 11:11:02 +02:00
a286ffa5f2 Alias macron package in app_routes.go
...to make this file compatible with goimports:
https://godoc.org/golang.org/x/tools/cmd/goimports
2017-10-11 08:18:03 +01:00
16c5d0e4b7 Always verify TLS unless explicitly told otherwise
TLS was not being verified in a number of places:

- connections to grafana.com

- connections to OAuth providers when TLS client authentication was
  enabled

- connections to self-hosted Grafana installations when using the CLI
  tool

TLS should always be verified unless the user explicitly enables an
option to skip verification.

Removes some instances where `InsecureSkipVerify` is explicitly set to
`false`, the default, to help avoid confusion and make it more difficult
to regress on this fix by accident.

Adds a `--insecure` flag to `grafana-cli` to skip TLS verification.

Adds a `tls_skip_verify_insecure` setting for OAuth.

Adds a `app_tls_skip_verify_insecure` setting under a new `[plugins]`
section.

I'm not super happy with the way the global setting is used by
`pkg/api/app_routes.go` but that seems to be the existing pattern used.
2017-10-06 17:09:27 +01:00
e9e8ffc769 Merge pull request #9367 from bobrik/dualstack
Enable dualstack in every net.Dialer, fixes #9364
2017-09-28 11:15:49 +02:00
9a4ae30227 Enable dualstack in every net.Dialer, fixes #9364
Default transport enables it:

* https://github.com/golang/go/blob/d2826d3e06/src/net/http/transport.go#L42-L46

```
    DialContext: (&net.Dialer{
	Timeout:   30 * time.Second,
	KeepAlive: 30 * time.Second,
	DualStack: true,
    }).DialContext,
```

See also: https://github.com/golang/go/issues/15324
2017-09-27 22:29:48 -07:00
8175783f93 go fmt 2017-09-24 18:59:21 +02:00
cffbcb504d allow ssl renegotiation for datasources 2017-09-13 16:20:24 -07:00
8bf49c51b9 dataproxy: refactoring data source proxy to support route templates and wrote more tests for data proxy code, #9078 2017-08-23 10:52:31 +02:00
56b7e2dfaf Added support for TLS client auth for datasource proxies (#5801) 2016-11-24 13:22:25 +01:00
e0c6048820 feat(instrumentation): added gauge and http endpoint 2016-06-03 16:15:36 +02:00
37c6a1ddf0 feat(app routes): worked on app routes, added unit test, changed Grafana-Context header to start with X to be standard compliant, got cloud saas queries to work via app route feature and header template 2016-02-10 16:43:35 +01:00
32f78d465b add secureJsonData to appSettings model.
- adds the new column to the DB table.
- data stored in the DB is encrypted
- update appRouteHeaders templates to use the jsonData and
decrypted secureJsonData
2016-01-23 06:17:22 +08:00
f94599cd29 merge apiPlugins with appPlugins 2016-01-22 01:15:04 +08:00