208 Commits

Author SHA1 Message Date
3c218d742f Datasources: Add user_agent header customization for outgoing HTTP requests (#63769) 2023-02-28 14:10:05 +01:00
23c27cffb3 Chore: Rename Id to ID in alerting models (#62777)
* Chore: Rename Id to ID in alerting models

* Add xorm tags for datasource

* Add xorm tag for uid
2023-02-02 17:22:43 +01:00
317f044e70 Plugins: Ensure proxy route bodies are valid JSON (#61771)
ensure proxy route bodies are valid JSON
2023-02-01 16:37:08 +01:00
9256a520a4 chore: move user_auth models to (mostly) login service (#62269)
* chore: move user_auth models to (mostly) login service
2023-01-27 13:36:54 -05:00
6c5a573772 Chore: Move ReqContext to contexthandler service (#62102)
* Chore: Move ReqContext to contexthandler service

* Rename package to contextmodel

* Generate ngalert files

* Remove unused imports
2023-01-27 08:50:36 +01:00
6478d0a5ef Plugins: Forward user header (X-Grafana-User) in backend plugin requests (#58646)
Grafana would forward the X-Grafana-User header to backend plugin request when 
dataproxy.send_user_header is enabled. In addition, X-Grafana-User will be automatically
forwarded in outgoing HTTP requests for core/builtin HTTP datasources. 
Use grafana-plugin-sdk-go v0.147.0.

Fixes #47734

Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-12-15 15:28:25 +01:00
f0adc69ada Chore: Change fmt.Errorf to errors.New when there is no formatting required (#58600)
Signed-off-by: Sasha Melentyev <sasha@melentyev.io>
2022-12-01 20:51:12 +01:00
9855e74b92 Chore: Refactor quota service (#58643)
Chore: Refactor quota service (#57586)

* Chore: refactore quota service

* Apply suggestions from code review
2022-11-14 21:08:10 +02:00
96cdf77995 Revert "Chore: Refactor quota service (#57586)" (#58394)
This reverts commit 326ea86a579ed927b1999bba5f2c0a35e26506d9.
2022-11-08 11:52:07 +02:00
326ea86a57 Chore: Refactor quota service (#57586)
* Chore: refactore quota service

* Apply suggestions from code review
2022-11-08 10:25:34 +02:00
6f8fcae01b [main] Plugin fixes (#57399)
* Plugins: Remove support for V1 manifests

* Plugins: Make proxy endpoints not leak sensitive HTTP headers

* Security: Fix do not forward login cookie in outgoing requests

(cherry picked from commit 4539c33fce5ef23badb08ebcbc09cb0cecb1f539)

Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-10-21 13:54:55 +02:00
05709ce411 chore: remove sqlstore & mockstore dependencies from (most) packages (#57087)
* chore: add alias for InitTestDB and Session

Adds an alias for the sqlstore InitTestDB and Session, and updates tests using these to reduce dependencies on the sqlstore.Store.

* next pass of removing sqlstore imports
* last little bit
* remove mockstore where possible
2022-10-19 09:02:15 -04:00
9c954d06ab Auth: Refresh OAuth access_token automatically using the refresh_token (#56076)
* Verify OAuth token expiration for oauth users in the ctx handler middleware

* Use refresh token to get a new access token

* Refactor oauth_token.go

* Add tests for the middleware changes

* Align other tests

* Add tests, wip

* Add more tests

* Add InvalidateOAuthTokens method

* Fix ExpiryDate update to default

* Invalidate OAuth tokens during logout

* Improve logout

* Add more comments

* Cleanup

* Fix import order

* Add error to HasOAuthEntry return values

* add dev debug logs

* Fix tests

Co-authored-by: jguer <joao.guerreiro@grafana.com>
2022-10-18 18:17:28 +02:00
862a6a2fa6 Logging: Introduce API for contextual logging (#55198)
Introduces a FromContext method on the log.Logger interface that 
allows contextual key/value pairs to be attached, e.g. per request, 
so that any logger using this API will automatically get the per request 
context attached. The proposal makes the traceID available for 
contextual logger , if available, and would allow logs originating from 
a certain HTTP request to be correlated with traceID.
In addition, when tracing not enabled, skip adding
traceID=00000000000000000000000000000000
to logs.
2022-09-20 18:32:06 +02:00
f25c7f6ddd Chore: Refactor secrets kvstore to organize testing and migrations (#54249)
* Refactor migrations and tests for secrets kvstore

* Use fake secrets store as a shortcut on tests

* Update wire

* Use global migration logger

* Fix ds proxy tests

* Fix linting issues

* Rename data source test setup function
2022-08-25 18:04:44 -03:00
e6857bf17d Plugins: Handle app plugin proxy routes per request (#51835)
Fixes #47530
2022-08-23 13:05:31 +02:00
a14621fff6 Chore: Add user service method SetUsingOrg and GetSignedInUserWithCacheCtx (#53343)
* Chore: Add user service method SetUsingOrg

* Chore: Add user service method GetSignedInUserWithCacheCtx

* Use method GetSignedInUserWithCacheCtx from user service

* Fix lint after rebase

* Fix lint

* Fix lint error

* roll back some changes

* Roll back changes in api and middleware

* Add xorm tags to SignedInUser ID fields
2022-08-11 13:28:55 +02:00
Jo
062d255124 Handle ioutil deprecations (#53526)
* replace ioutil.ReadFile -> os.ReadFile

* replace ioutil.ReadAll -> io.ReadAll

* replace ioutil.TempFile -> os.CreateTemp

* replace ioutil.NopCloser -> io.NopCloser

* replace ioutil.WriteFile -> os.WriteFile

* replace ioutil.TempDir -> os.MkdirTemp

* replace ioutil.Discard -> io.Discard
2022-08-10 15:37:51 +02:00
6afad51761 Move SignedInUser to user service and RoleType and Roles to org (#53445)
* Move SignedInUser to user service and RoleType and Roles to org

* Use go naming convention for roles

* Fix some imports and leftovers

* Fix ldap debug test

* Fix lint

* Fix lint 2

* Fix lint 3

* Fix type and not needed conversion

* Clean up messages in api tests

* Clean up api tests 2
2022-08-10 11:56:48 +02:00
945f015770 backend/datasources: move datasources models into the datasources service package (#51267)
* backend/datasources: move datasources models into the datasources service pkg
2022-06-27 12:23:15 -04:00
ae9491c3a7 Chore: Make test tracer noop and return no errors (#50797) 2022-06-15 12:40:41 +02:00
36c3398c6d Datasource: Remove support for unencrypted passwords (#49987)
* Datasource: Remove support for unencrypted passwords

* regenerate swagger

* [WIP] Remove references to datasource password and basic auth password fields (#50015)

* try delete moar tings

* delete provisioning stuff

* remove from yaml

* update snapshots

* remove lingering snapshot fields

* fix ds http settings

* Re-generate swagger and fix swagger-api-spec make target

Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-06-03 17:38:22 +02:00
a7813275a5 Plugins: Support headers field for check health (#49930) 2022-05-31 17:58:06 +02:00
cc0448927a Add TraceID to data proxy logs (#49302) 2022-05-23 10:38:05 +02:00
61772a66b6 AccessControl: Create own interface and impl for each permission service (#48871)
* Create own interfaces for team, folder, dashboard and data source permissions services
* Remove service container and inject them individually
2022-05-10 15:48:47 +02:00
a367ad730c Secrets: Implement basic unified secret store service (#45804)
* wip: Implement kvstore for secrets

* wip: Refactor kvstore for secrets

* wip: Add format key function to secrets kvstore sql

* wip: Add migration for secrets kvstore

* Remove unused Key field from secrets kvstore

* Remove secret values from debug logs

* Integrate unified secrets with datasources

* Fix minor issues and tests for kvstore

* Create test service helper for secret store

* Remove encryption tests from datasources

* Move secret operations after datasources

* Fix datasource proxy tests

* Fix legacy data tests

* Add Name to all delete data source commands

* Implement decryption cache on sql secret store

* Fix minor issue with cache and tests

* Use secret type on secret store datasource operations

* Add comments to make create and update clear

* Rename itemFound variable to isFound

* Improve secret deletion and cache management

* Add base64 encoding to sql secret store

* Move secret retrieval to decrypted values function

* Refactor decrypt secure json data functions

* Fix expr tests

* Fix datasource tests

* Fix plugin proxy tests

* Fix query tests

* Fix metrics api tests

* Remove unused fake secrets service from query tests

* Add rename function to secret store

* Add check for error renaming secret

* Remove bus from tests to fix merge conflicts

* Add background secrets migration to datasources

* Get datasource secure json fields from secrets

* Move migration to secret store

* Revert "Move migration to secret store"

This reverts commit 7c3f872072e9aff601fb9d639127d468c03f97ef.

* Add secret service to datasource service on tests

* Fix datasource tests

* Remove merge conflict on wire

* Add ctx to data source http transport on prometheus stats collector

* Add ctx to data source http transport on stats collector test
2022-04-25 13:57:45 -03:00
e86b6662a1 Chore: Remove bus.Bus field (#47695)
* Chore: Remove bus.Bus field

* fix integration test
2022-04-13 15:24:13 +02:00
4bc582570e Instrumentation: Proxy status code correction and various improvements (#47473)
For a proxied request, e.g. Grafana's datasource or plugin proxy:
If the request is cancelled, e.g. from the browser, the HTTP status code is 
now 499 Client closed request instead of 502 Bad gateway.
If the request times out, e.g. takes longer time than allowed, the HTTP status 
code is now 504 Gateway timeout instead of 502 Bad gateway.
This also means that request metrics and logs will get their status codes 
adjusted according to above.

Fixes #46337
Fixes #46338
2022-04-11 13:17:08 +02:00
18e93c7077 Chore: Remove bus (#47511)
* Chore: Remove bus

* remove unused const
2022-04-08 16:15:06 +02:00
5675496f6b Migrate to Grafana Azure SDK (#47232) 2022-04-04 11:23:13 +02:00
656ade9884 Shared Azure middleware between Azure Monitor and Prometheus datasources (#46002)
* Scopes in Azure middleware

* Enable Azure middleware without feature flag

* Use common Azure middleware in Azure Monitor

* Apply feature flag to JsonData configuration of Azure auth

* Enforce feature flag in Prometheus datasource

* Prometheus provider tests

* Datasource service tests

* Fix http client provider tests

* Pass sdkhttpclient.Options by reference

* Add middleware to httpclient.Options

* Remove dependency on Grafana settings

* Unit-tests updated

* Fix ds_proxy_test

* Fix service_test
2022-04-01 13:26:49 +02:00
c717320942 Adding FGAC annotations validation for creation and deletion (#46736)
Access Control: Adding FGAC annotations validation for creation and deletion
Co-authored-by: IevaVasiljeva <ieva.vasiljeva@grafana.com>
2022-03-21 14:28:39 -03:00
bda3f860a8 Plugins: Add plugin settings DTO (#46283)
* add clearer service layer

* re-order frontend settings for clarity

* fix fetch fail

* fix API response

* fix mockstore

* in -> where
2022-03-18 20:49:13 +01:00
6a8cbd8663 Plugins: Plugin settings refactor (#46246)
* remove bus and direct use of sqlStore

* add decryption to interface

* return nil

* rename field

* re-order fields

* rename file
2022-03-04 17:09:50 +01:00
b54b438a24 Plugins: Refactor plugin settings service (#45967)
* tests passing

* rename and rejig

* move interface to package and rename to Store

* new package

* add import alias
2022-03-03 11:39:15 +01:00
2334b98802 Use PluginSettingsService instead of SQLStore methods in plugins (#45480)
* Use PluginSettingsService instead of SQLStore in plugins

* Fix pluginproxy use of pluginsettings methods

* Fix additional pluginsettings methods

* Remove dispatch from plugindashboards

* Fix lint and adjust mock

* Remove unused pluginsettings

* Rename pluginsetting Service and ServiceImpl and add binding to wire

* Move pluginsettings binding in wire file
2022-02-25 11:29:18 +01:00
cdc08105c2 Access control: Set default permissions for data sources when using access control (#45482)
* Rename interfaces and use then with wire injection

* Set default permissions when creating new data source
2022-02-17 14:03:45 +01:00
6a776c78d4 Datasource: Fixes storing of secureJSONData when creating/updating datasource (#45290)
Fixes an issue introduced by #44987 where bus dispatch was replaced by calling sqlstore 
directly instead of the datasource service.

Fixes #45273
2022-02-11 15:52:14 +01:00
605d056136 Security: Sync security changes on main (#45083)
* * Teams: Appropriately apply user id filter in /api/teams/:id and /api/teams/search
* Teams: Ensure that users searching for teams are only able see teams they have access to
* Teams: Require teamGuardian admin privileges to list team members
* Teams: Prevent org viewers from administering teams
* Teams: Add org_id condition to team count query
* Teams: clarify permission requirements in teams api docs
* Teams: expand scenarios for team search tests
* Teams: mock teamGuardian in tests

Co-authored-by: Dan Cech <dcech@grafana.com>

* remove duplicate WHERE statement

* Fix for CVE-2022-21702

(cherry picked from commit 202d7c190082c094bc1dc13f7fe9464746c37f9e)

* Lint and test fixes

(cherry picked from commit 3e6b67d5504abf4a1d7b8d621f04d062c048e981)

* check content type properly

(cherry picked from commit 70b4458892bf2f776302720c10d24c9ff34edd98)

* basic csrf origin check

(cherry picked from commit 3adaa5ff39832364f6390881fb5b42ad47df92e1)

* compare origin to host

(cherry picked from commit 5443892699e8ed42836bb2b9a44744ff3e970f42)

* simplify url parsing

(cherry picked from commit b2ffbc9513fed75468628370a48b929d30af2b1d)

* check csrf for GET requests, only compare origin

(cherry picked from commit 8b81dc12d8f8a1f07852809c5b4d44f0f0b1d709)

* parse content type properly

(cherry picked from commit 16f76f4902e6f2188bea9606c68b551af186bdc0)

* mentioned get in the comment

(cherry picked from commit a7e61811ef8ae558ce721e2e3fed04ce7a5a5345)

* add content-type: application/json to test HTTP requests

* fix pluginproxy test

* Fix linter when comparing errors

Co-authored-by: Kevin Minehart <kmineh0151@gmail.com>
Co-authored-by: Dan Cech <dcech@grafana.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Serge Zaitsev <serge.zaitsev@grafana.com>
Co-authored-by: Vardan Torosyan <vardants@gmail.com>
2022-02-09 13:44:38 +01:00
9d654bb6b8 Chore: Remove bus from plugin proxy api (#44899)
* remove bus from plugin proxy api

* fixing the tests
2022-02-04 17:35:00 +01:00
7ed82ac049 Check for empty URLs when creating/updating a data source (#42837)
* checks for empty URLs added

* check for TimeSeriesTypeNot to fix InfluxDB alerts

* log a warning when a data frame is ignored

* fix: add brittle Prometheus URL input selector

needs a proper aria-label or test-data-id selector

* test: add URL input aria-label

needs to use the grafana/e2e-selectors package

* test: run ci

* add URL validation for specific data sources, e2e tests

* Update pkg/api/datasource/validation.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* delete duplicated logs

* delete unnecessary leading newline

Co-authored-by: gillesdemey <gilles.de.mey@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-01-31 12:39:55 -03:00
dbb2d3af73 Chore: Fix attribute value (#44368)
* Fix attribute value

* Fix also the value for opentracing
2022-01-24 18:17:39 +01:00
30aa24a183 Chore: Implement OpenTelemtry in Grafana (#42674)
* Separate Tracer interface to TracerService and Tracer

* Fix lint

* Fix:Make it possible to start spans for both opentracing and opentelemetry in ds proxy

* Add span methods, use span interface for rest of tracing

* Fix logs in tracing

* Fix tests that are related to tracing

* Fix resourcepermissions test

* Fix some tests

* Fix more tests

* Add TracingService to wire cli runner

* Remove GlobalTracer from bus

* Renaming test function

* Remove GlobalTracer from TSDB

* Replace GlobalTracer in api

* Adjust tests to the InitializeForTests func

* Remove GlobalTracer from services

* Remove GlobalTracer

* Remove bus.NewTest

* Remove Tracer interface

* Add InitializeForBus

* Simplify tests

* Clean up tests

* Rename TracerService to Tracer

* Update pkg/middleware/request_tracing.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Initialize tracer before passing it to SQLStore initialization in commands

* Remove tests for opentracing

* Set span attributes correctly, remove unnecessary trace initiliazation form test

* Add tracer instance to newSQLStore

* Fix changes due to rebase

* Add modified tracing middleware test

* Fix opentracing implementation tags

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-01-20 11:10:12 +01:00
54280fc9d7 AccessControl: Resolve attribute based scopes to id based scopes (#40742)
* AccessControl: POC scope attribute resolution

Refactor based on ScopeMutators

test errors and calls to cache

Add comments to tests

Rename logger

Create keywordMutator only once

* AccessControl: Add AttributeScopeResolver registration

Co-authored-by: gamab <gabriel.mabille@grafana.com>

* AccessControl: Add AttributeScopeResolver to datasources

Co-authored-by: gamab <gabriel.mabille@grafana.com>

* Test evaluation with translation

* fix imports

* AccessControl: Test attribute resolver

* Fix trailing white space

* Make ScopeResolver public for enterprise redefine

* Handle wildcard

Co-authored-by: Jguer <joao.guerreiro@grafana.com>

Co-authored-by: jguer <joao.guerreiro@grafana.com>
2022-01-18 17:34:35 +01:00
8e6d6af744 Rename DispatchCtx to Dispatch (#43563) 2021-12-28 17:36:22 +01:00
7936c4c522 Rename AddHandlerCtx to AddHandler (#43557) 2021-12-28 16:08:07 +01:00
f5802878f1 Chore: Move remaining web framework code to pkg/web, remove macaron binding module (#43018)
* remove macaron binding dependency

* completely purge macaron binding

* move everything to pkg/web

* remove non-go files from pkg/web

* clean up leftovers of macaron imports

* make linter happy
2021-12-13 15:56:14 +01:00
e6123bc3ef Replace AddHandler with AddHandlerCtx in tests (#42585) 2021-12-01 15:43:31 +01:00
becfd776c3 OAuth: Forward id token to the data source (#42422)
* OAuth: Forward id token to the data source

* Add tests

* Forward id token in legacy API

* Check if id_token is string or not
2021-11-29 15:40:05 +01:00
5652bde447 Encryption: Use secrets service (#40251)
* Use secrets service in pluginproxy

* Use secrets service in pluginxontext

* Use secrets service in pluginsettings

* Use secrets service in provisioning

* Use secrets service in authinfoservice

* Use secrets service in api

* Use secrets service in sqlstore

* Use secrets service in dashboardshapshots

* Use secrets service in tsdb

* Use secrets service in datasources

* Use secrets service in alerting

* Use secrets service in ngalert

* Break cyclic dependancy

* Refactor service

* Break cyclic dependancy

* Add FakeSecretsStore

* Setup Secrets Service in sqlstore

* Fix

* Continue secrets service refactoring

* Fix cyclic dependancy in sqlstore tests

* Fix secrets service references

* Fix linter errors

* Add fake secrets service for tests

* Refactor SetupTestSecretsService

* Update setting up secret service in tests

* Fix missing secrets service in multiorg_alertmanager_test

* Use fake db in tests and sort imports

* Use fake db in datasources tests

* Fix more tests

* Fix linter issues

* Attempt to fix plugin proxy tests

* Pass secrets service to getPluginProxiedRequest in pluginproxy tests

* Fix pluginproxy tests

* Revert using secrets service in alerting and provisioning

* Update decryptFn in alerting migration

* Rename defaultProvider to currentProvider

* Use fake secrets service in alert channels tests

* Refactor secrets service test helper

* Update setting up secrets service in tests

* Revert alerting changes in api

* Add comments

* Remove secrets service from background services

* Convert global encryption functions into vars

* Revert "Convert global encryption functions into vars"

This reverts commit 498eb19859eba364a2400a6d7e73236b1c9a5b37.

* Add feature toggle for envelope encryption

* Rename toggle

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
Co-authored-by: Joan López de la Franca Beltran <joanjan14@gmail.com>
2021-11-04 18:47:21 +02:00