* define initial service and add to wire
* update caching service interface
* add skipQueryCache header handler and update metrics query function to use it
* add caching service as a dependency to query service
* working caching impl
* propagate cache status to frontend in response
* beginning of improvements suggested by Lean - separate caching logic from query logic.
* more changes to simplify query function
* Decided to revert renaming of function
* Remove error status from cache request
* add extra documentation
* Move query caching duration metric to query package
* add a little bit of documentation
* wip: convert resource caching
* Change return type of query service QueryData to a QueryDataResponse with Headers
* update codeowners
* change X-Cache value to const
* use resource caching in endpoint handlers
* write resource headers to response even if it's not a cache hit
* fix panic caused by lack of nil check
* update unit test
* remove NONE header - shouldn't show up in OSS
* Convert everything to use the plugin middleware
* revert a few more things
* clean up unused vars
* start reverting resource caching, start to implement in plugin middleware
* revert more, fix typo
* Update caching interfaces - resource caching now has a separate cache method
* continue wiring up new resource caching conventions - still in progress
* add more safety to implementation
* remove some unused objects
* remove some code that I left in by accident
* add some comments, fix codeowners, fix duplicate registration
* fix source of panic in resource middleware
* Update client decorator test to provide an empty response object
* create tests for caching middleware
* fix unit test
* Update pkg/services/caching/service.go
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
* improve error message in error log
* quick docs update
* Remove use of mockery. Update return signature to return an explicit hit/miss bool
* create unit test for empty request context
* rename caching metrics to make it clear they pertain to caching
* Update pkg/services/pluginsintegration/clientmiddleware/caching_middleware.go
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Add clarifying comments to cache skip middleware func
* Add comment pointing to the resource cache update call
* fix unit tests (missing dependency)
* try to fix mystery syntax error
* fix a panic
* Caching: Introduce feature toggle to caching service refactor (#66323)
* introduce new feature toggle
* hide calls to new service behind a feature flag
* remove licensing flag from toggle (misunderstood what it was for)
* fix unit tests
* rerun toggle gen
---------
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Remove Result field from AddDataSourceCommand
* Remove DatasourcesPermissionFilterQuery Result
* Remove GetDataSourceQuery Result
* Remove GetDataSourcesByTypeQuery Result
* Remove GetDataSourcesQuery Result
* Remove GetDefaultDataSourceQuery Result
* Remove UpdateDataSourceCommand Result
Adding support for backend plugin client middlewares. This allows headers in outgoing
backend plugin and HTTP requests to be modified using client middlewares.
The following client middlewares added:
Forward cookies: Will forward incoming HTTP request Cookies to outgoing plugins.Client
and HTTP requests if the datasource has enabled forwarding of cookies (keepCookies).
Forward OAuth token: Will set OAuth token headers on outgoing plugins.Client and HTTP
requests if the datasource has enabled Forward OAuth Identity (oauthPassThru).
Clear auth headers: Will clear any outgoing HTTP headers that was part of the incoming
HTTP request and used when authenticating to Grafana.
The current suggested way to register client middlewares is to have a separate package,
pluginsintegration, responsible for bootstrap/instantiate the backend plugin client with
middlewares and/or longer term bootstrap/instantiate plugin management.
Fixes#54135
Related to #47734
Related to #57870
Related to #41623
Related to #57065
* Plugins: Remove support for V1 manifests
* Plugins: Make proxy endpoints not leak sensitive HTTP headers
* Security: Fix do not forward login cookie in outgoing requests
(cherry picked from commit 4539c33fce5ef23badb08ebcbc09cb0cecb1f539)
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
* Chore: Add user service method SetUsingOrg
* Chore: Add user service method GetSignedInUserWithCacheCtx
* Use method GetSignedInUserWithCacheCtx from user service
* Fix lint after rebase
* Fix lint
* Fix lint error
* roll back some changes
* Roll back changes in api and middleware
* Add xorm tags to SignedInUser ID fields
* Move SignedInUser to user service and RoleType and Roles to org
* Use go naming convention for roles
* Fix some imports and leftovers
* Fix ldap debug test
* Fix lint
* Fix lint 2
* Fix lint 3
* Fix type and not needed conversion
* Clean up messages in api tests
* Clean up api tests 2
* update decrypt secrets function signature and add secrets error handling
* remove a couple instances of unnecessary logging since errors are properly handled now
* add unit test
* fix linting issues
* Add protobuf config and generated code, and client wrapper
* wire up loading of secretsmanager plugin, using renderer plugin as a model
* update kvstore provider to check if we should use the grpc plugin. return false always in OSS
* add OSS remote plugin check
* refactor wire gen file
* log which secrets manager is being used
* Fix argument types for remote checker
* Turns out if err != nil, then the result is always nil. Return empty values if there is an error.
* remove duplicate import
* ensure atomicity by adding secret management as a step to sql operations and rolling back if necessary
* Update pkg/services/secrets/kvstore/kvstore.go
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Update pkg/services/secrets/kvstore/kvstore.go
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* refactor RemotePluginCheck interface to just return the Plugin client directly
* rename struct to something less silly
* add special error handling for remote secrets management
* switch to errors.as instead of type inference
* remove unnecessary rollback call
* just declare error once
* refactor .proto file according to prior PR suggestions
* re-generate protobuf files and fix compilation errors
* only wrap (ergo display in the front end) errors that are user friendly from the plugin
* rename error type to suggest user friendly only
* rename plugin functions to be more descriptive
* change delete message name
* Revert "change delete message name"
This reverts commit 8ca978301eea45d3cc6a22cda15b9d099c533955.
* Revert "rename plugin functions to be more descriptive"
This reverts commit 4355c9b9ff95443f74c0b588d1ffeabb544f1a34.
* fix pointer to pointer problem
* change plugin user error to just hold a string
* fix sequencing problem with datasource updates
* clean up some return statements
* need to wrap multiple transactions with the InTransaction() func in order to keep the lock
* make linter happy
* revert input var name
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Fix secure json data reset on datasource update
* Update fillWithSecureJSONData to use DecryptedValues
* Remove unecessary conversion
* Move fillWithSecureJsonData logic to datasource service
* Add sanity check for nil secure json data
* Add test for decrypted values on datasource service
* Add debug log when fail to parse secure json fields
* Fix minor import issue
* Refactor encJson to json and simplejson to sjson on tests
* wip: Implement kvstore for secrets
* wip: Refactor kvstore for secrets
* wip: Add format key function to secrets kvstore sql
* wip: Add migration for secrets kvstore
* Remove unused Key field from secrets kvstore
* Remove secret values from debug logs
* Integrate unified secrets with datasources
* Fix minor issues and tests for kvstore
* Create test service helper for secret store
* Remove encryption tests from datasources
* Move secret operations after datasources
* Fix datasource proxy tests
* Fix legacy data tests
* Add Name to all delete data source commands
* Implement decryption cache on sql secret store
* Fix minor issue with cache and tests
* Use secret type on secret store datasource operations
* Add comments to make create and update clear
* Rename itemFound variable to isFound
* Improve secret deletion and cache management
* Add base64 encoding to sql secret store
* Move secret retrieval to decrypted values function
* Refactor decrypt secure json data functions
* Fix expr tests
* Fix datasource tests
* Fix plugin proxy tests
* Fix query tests
* Fix metrics api tests
* Remove unused fake secrets service from query tests
* Add rename function to secret store
* Add check for error renaming secret
* Remove bus from tests to fix merge conflicts
* Add background secrets migration to datasources
* Get datasource secure json fields from secrets
* Move migration to secret store
* Revert "Move migration to secret store"
This reverts commit 7c3f872072e9aff601fb9d639127d468c03f97ef.
* Add secret service to datasource service on tests
* Fix datasource tests
* Remove merge conflict on wire
* Add ctx to data source http transport on prometheus stats collector
* Add ctx to data source http transport on stats collector test
* Add ResourceAttribute
* Add ResourceAttribute option
* Set ResourceAttribute option
* Change resolvers to return uid based scopes
* update swagger to correct scope
* use ResourceAttribute for endpoint scope
* bump role version
* Add support for different attributes for access control metadata
* evaluate data source metadata based on uid
* Fix test
* uncomment benchmarks
* Use resourceID
* use evaluator for access control metadata
* update comment
* Set default permissions based on uid
* Add attribute to accesscontrol filter
* validate that scopes has correct attribute
* lint
* Update comment
* remove attribute parameter and extend prefix
* refactor to use scope prefix
* Get metadata with prefix
* fix test
* fix comparision
* remove unused type
* fix attribute index
* fix typo
* restructure logic
* Get metadata by uid
* fix imports
Co-authored-by: jguer <joao.guerreiro@grafana.com>
* Add option to set ResourceAttribute for a permissions service
* Use prefix in access control sql filter to parse scopes
* Use prefix in access control metadata to check access
* Data sources: Sent user ID when creating data source
* Data sources: Grant a data source creator edit permissions
* Use edit permisison and only append if user id is in command
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
* AccessControl: Add endpoint to get user permissions
Co-authored-by: ievaVasiljeva <ieva.vasiljeva@grafana.com>
Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
Co-authored-by: Eric Leijonmarck <eric.leijonmarck@gmail.com>
Co-authored-by: Alexander Zobnin <alexanderzobnin@gmail.com>
* Fix SA tests
* Linter is wrong :p
* Wait I was wrong
* Adding the route for teams:creator too
Co-authored-by: ievaVasiljeva <ieva.vasiljeva@grafana.com>
Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
Co-authored-by: Eric Leijonmarck <eric.leijonmarck@gmail.com>
Co-authored-by: Alexander Zobnin <alexanderzobnin@gmail.com>
* Remove bus from datasource api
* Add DatasourcePermissionService and use it in api
* Fix wire and rename
* Fix import in wire
* Fix bug
* Rename Service to OSS service
* Roll back fix
* checks for empty URLs added
* check for TimeSeriesTypeNot to fix InfluxDB alerts
* log a warning when a data frame is ignored
* fix: add brittle Prometheus URL input selector
needs a proper aria-label or test-data-id selector
* test: add URL input aria-label
needs to use the grafana/e2e-selectors package
* test: run ci
* add URL validation for specific data sources, e2e tests
* Update pkg/api/datasource/validation.go
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* delete duplicated logs
* delete unnecessary leading newline
Co-authored-by: gillesdemey <gilles.de.mey@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Check if datasource is read-only when making an update
* Standardize api returning a 404 if datasource is not found while making an update
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>, Jesse Weaver<pianohacker@gmail.com>
* AccessControl: Provide scope to frontend
* Covering datasources with accesscontrol metadata
* Write benchmark tests for GetResourcesMetadata
* Add accesscontrol util and interface
* Add the hasPermissionInMetadata function in the frontend access control code
* Use IsDisabled rather that performing a feature toggle check
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
* Chore: Refactor api handlers to use web.Bind
* fix comments
* fix comment
* trying to fix most of the tests and force routing.Wrap type check
* fix library panels tests
* fix frontend logging tests
* allow passing nil as a response to skip writing
* return nil instead of the response
* rewrite login handler function types
* remove handlerFuncCtx
* make linter happy
* remove old bindings from the libraryelements
* restore comments
