175 Commits

Author SHA1 Message Date
701f6d5436 UserService: use the UserService instead of calling sqlstore directly (#55745)
* UserService: update callers to use the UserService instead of calling sqlstore directly

There is one major change hiding in this PR. UserService.Delete originally called a number of services to delete user-related records. I moved everything except the actual call to the user table, and moved those into the API. This was done to avoid dependencies cycles; many of our services depend on the user service, so the user service itself should have as few dependencies as possible.
2022-09-27 07:58:49 -04:00
55aae79744 GRPC Server: Add gRPC server service (#47849)
Co-authored-by: Todd Treece <todd.treece@grafana.com>
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2022-09-26 16:25:34 -04:00
d0d8544ded Plugins: Create single point of entry for adding / removing plugins (#55463)
* split out plugin manager

* remove whitespace

* fix tests

* split up tests

* updating naming conventions

* simplify manager

* tidy

* explorations

* fix build

* tidy

* fix tests

* add logger helper

* pass the tests

* tidying

* fix tests

* tidy and re-add test

* store depends on loader

* enrich tests

* fix test

* undo gomod changes
2022-09-23 14:27:01 +02:00
f8d69415ca Search: create a separate HTTP endpoint (#55634)
* search: create a separate http endpoint

* search: extract api uri

* search: rename uri

* search: replicate the readiness check

* search: replicate the readiness check metric

* search: update mock
2022-09-22 19:02:09 -04:00
09f4068849 NavTree: Refactor out the navtree building from api/index.go and into it's own service (#55552) 2022-09-22 22:04:48 +02:00
4c19e83ff0 Chore: Move team store implementation to a separate package (#55514)
* Chore: move team store implementation to a separate package

* trying to fix more tests

* fix tests in service accounts and access control

* fix common tests

* restore commented out test

* add todos
2022-09-22 19:16:21 +02:00
2f14575dd3 Chore: Move annotations cleanup to the annotations service (#55618) 2022-09-22 15:27:48 +03:00
7b4cea8151 Chore: StoreSplit tag service (#55453)
* move tag service outside

* fix dashboard

* fix test

* lint

* fix linter

* remove spew
2022-09-21 08:04:01 -04:00
2ee3fd3994 Chore: Create structure for team service (#55444) 2022-09-20 10:14:00 +00:00
8f1e2ed658 Chore: Split temporary user (invite) service (#55397)
* Chore: Split temporary users

* change references to tempuser service

* fix api tests

* restore tests
2022-09-20 11:29:17 +02:00
754eea20b3 Chore: SQL store split for annotations (#55089)
* Chore: SQL store split for annotations

* Apply suggestion from code review
2022-09-19 10:54:37 +03:00
b827df626d RBAC: Initiate store in service (#55081)
* RBAC: Dont inject store with wire

* RBAC: Use Store interface

* RBAC: Move store interface and initiate it from service
2022-09-15 11:34:15 +02:00
0571d98bba [WIP] Plugins: Introduce Plugins specific config (#54854) 2022-09-14 14:35:35 +01:00
20589f76f4 Secrets: Convert secret migration to a background service (#54676)
* Convert secret migration to a background service

* Fix merge conflicts

* Return concrete type for secret migration provider
2022-09-06 16:21:25 -03:00
ff35e35ce0 RBAC: Move service and evaluator to acimpl package (#54714)
* RBAC: Move access control evaluator to acimpl package

* RBAC: Move service to acimpl package
2022-09-05 18:15:47 +02:00
ecdcafb258 manager exposes renderer + secrets manager (#54629) 2022-09-02 14:20:10 +02:00
927ddf9376 Chore: Move login attempt methods to separate service (#54479)
* Chore: Move login attempt methods to separate service

* attempt to fix tests

* fix syntax

* better time mocking

* initialise now func
2022-09-01 18:08:42 +02:00
58449d42ed Chore: Add dashboard thumbnails service (#54500)
* Chore: Add dashboard thumbnails service

* Fix errors in dashboard thumbs impl

* Inject dashboardThumbsService into wire
2022-08-31 16:45:41 -04:00
166b981019 ensure process manager is background service (#54475) 2022-08-30 13:09:52 -04:00
4a707e2a88 Plugins: Split plugin manager into smaller components (#54384)
* split out plugin manager

* remove whitespace

* fix tests

* split up tests

* updating naming conventions

* simplify manager

* tidy

* add more fakes

* testing time

* add query verif to int test

* renaming

* add process tests

* tidy up manager tests

* add extra case to int test

* add more coverage to store and process tests

* remove comment

* fix capatilization

* init on provide

* remove addfromsource from API
2022-08-30 17:30:43 +02:00
f25c7f6ddd Chore: Refactor secrets kvstore to organize testing and migrations (#54249)
* Refactor migrations and tests for secrets kvstore

* Use fake secrets store as a shortcut on tests

* Update wire

* Use global migration logger

* Fix ds proxy tests

* Fix linting issues

* Rename data source test setup function
2022-08-25 18:04:44 -03:00
277ea836b6 Secrets: Implement migration of secrets from plugin back to unified secrets (#53561)
* initial cut at migration from plugin

* create new migration from plugin

* only migrate to or from, not both

* remove cfg check from plugin migration itself

* update comments, clean up secret after migration

* add better error handling

* hook up REST API with migrations

* Minor fixes

* fix wire injection issue

* modify migrator to access plugin calls directly. create unit tests

* change pre-migration checks in admin api

* stop plugin after migrating from it

* fix compile issues after merge

* add comment about migration

* fix linting issue

* bleh, fix unit test

* fix another unit test

* update plugin error fatal flag after a migration from the plugin

* add extra logging to migration

* make linter happy

Co-authored-by: Leandro Deveikis <leandro.deveikis@gmail.com>
2022-08-24 16:24:50 -04:00
55c7b8add2 RBAC: Split up service into several components (#54002)
* RBAC: Rename interface to Store

* RBAC: Move ranme scopeInjector

* RBAC: Rename files to service

* RBAC: Rename to service

* RBAC: Split up accesscontrol into two components

* RBAC: Add DeclareFixedRoles to AccessControl interface

* Wire: Fix wire bindings

* RBAC: Move resolvers to root

* RBAC: Remove invalid test

* RBAC: Inject access control service

* RBAC: Implement the RoleRegistry interface in fake
2022-08-24 13:29:17 +02:00
26dfdd5af3 Plugins: Refactor plugin download/installation (#43046)
* installer -> repo

* add semver format checking

* add plugin callbacks in test

* remove newline

* post install only scans new directories

* remove unused stuff

* everything in own package

* add missing cli params

* make grafana version part of the API

* resolve conflicts

* tidy up logger

* fix cli and tidy log statements

* rename log package

* update struct name

* fix linter issue

* fs -> filestore

* reorder imports

* alias import

* fix test

* fix test

* inline var

* revert jsonc file

* make repo dep of manager

* actually inject the thing

* accept all args for compatability checks

* accept compat from store

* pass os + arch vals

* don't inject fs

* tidy up

* tidy up

* merge with main and tidy fs storage

* fix test

* fix packages

* fix comment + field name

* update fs naming

* fixed wire

* remove unused func

* fix mocks

* fix storage test

* renaming

* fix log line

* fix test

* re-order field

* tidying

* add test for update with same version

* fix wire for CLI

* remove use of ioutil

* don't pass field

* small tidy

* ignore code scanning warn

* fix testdata link

* update lgtm code
2022-08-23 11:50:50 +02:00
1b933ff3ed RBAC: Move resource permissions store to service package (#53815)
* Rename file to store

* Move resource permission specific database functions to
resourcepermissions package

* Wire: Remove interface bind

* RBAC: Remove injection of resourcepermission Store

* RBAC: Export store constructor

* Tests: Use resource permission package to initiate store used in tests

* RBAC: Remove internal types package and move to resourcepermissions
package

* RBAC: Run database tests as itegration tests
2022-08-18 09:43:45 +02:00
7c5ddaea58 Chore: Add loginattempt service (#53687)
* Chore: Add loginattempt service

* Inject loginattemptservice into httpserver
2022-08-17 08:34:23 +02:00
943cdea855 Chore: Add tempuser service (#53325)
* Chore: Add tempuser service

* Add implementation

* Fix linter; Inject tempUser into wire and HTTPServer

* Fix errors
2022-08-12 12:13:23 -04:00
da72a4ed2e Chore move Filter to user service (#53588) 2022-08-11 14:45:29 +02:00
7924d3b3b5 Secrets: Move license check to secret store plugin (#53185)
* Move license check to secret store plugin

* Use shouldUseRemoteSecretsPlugin for migration

* Encapsulate plugin startup logic in sync.Once

* Remove global logger from startup functions

* Fix issues with wire and secrets plugin check

* Remove todo for plugin fatal error

* Rename fatalErr variable to be less confusing

* Fix merge conflicts

* Fix issue with grafana-cli wire and opentsdb

* Remove duplicated import on remote plugin

* Rename plugin check in favor of error return value

* Remove unnecessary import on grafana-cli wireexts_oss

* Remove unnecessary import on grafana wireexts_oss

* Reset sync.Once during test setup

* Remove unrelated opentsdb change on grafana-cli wire

* Readd opentsdb change on grafana-cli wire
2022-08-10 16:47:03 -03:00
Jo
062d255124 Handle ioutil deprecations (#53526)
* replace ioutil.ReadFile -> os.ReadFile

* replace ioutil.ReadAll -> io.ReadAll

* replace ioutil.TempFile -> os.CreateTemp

* replace ioutil.NopCloser -> io.NopCloser

* replace ioutil.WriteFile -> os.WriteFile

* replace ioutil.TempDir -> os.MkdirTemp

* replace ioutil.Discard -> io.Discard
2022-08-10 15:37:51 +02:00
196b781c70 Alerting: Delete expired images from the database (#53236)
This commit adds a DeleteExpiredService that deletes expired images from the database. It is run in the periodic collector service.
2022-08-09 15:28:36 +01:00
1ecbe22751 Chore: Add user service method GetByLogin (#53204)
* Add wrapper around sqlstore method GetUserByLogin

* Use new method from user service

* Fix lint

* Fix lint 2

* fix middleware basic auth test

* Fix grafana login returning a user by login

* Remove GetUserByLogin from store interface

* Merge commit
2022-08-04 13:22:43 +02:00
643d2bc890 Chore: Sql store split for legacy alerting (#52901)
Moves ~20 sqlstore methods for legacy alerting out of sqlstore (sqlstore.Store interface) and into alerting.
2022-08-03 11:17:26 -04:00
fab6c38c95 Chore: Split get user by ID (#52442)
* Remove user from preferences, stars, orguser, team member

* Fix lint

* Add Delete user from org and dashboard acl

* Delete user from user auth

* Add DeleteUser to quota

* Add test files and adjust user auth store

* Rename package in wire for user auth

* Import Quota Service interface in other services

* do the same in tests

* fix lint tests

* Fix tests

* Add some tests

* Rename InsertUser and DeleteUser to InsertOrgUser and DeleteOrgUser

* Rename DeleteUser to DeleteByUser in quota

* changing a method name in few additional places

* Fix in other places

* Fix lint

* Fix tests

* Chore: Split Delete User method

* Add fakes for userauth

* Add mock for access control Delete User permossion, use interface

* Use interface for ream guardian

* Add simple fake for dashboard acl

* Add go routines, clean up, use interfaces

* fix lint

* Update pkg/services/user/userimpl/user_test.go

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* Update pkg/services/user/userimpl/user_test.go

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* Update pkg/services/user/userimpl/user_test.go

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* Split get user by ID

* Use new method in api

* Add tests

* Aplly emthod in auth info service

* Fix lint and some tests

* Fix get user by ID

* Fix lint
Remove unused fakes

* Use split get user id in admin users

* Use GetbyID in cli commands

* Clean up after merge

* Remove commented out code

* Clena up imports

* add back )

* Fix wire generation for runner after merge with main

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
2022-08-02 16:58:05 +02:00
64488f6b90 Chore: split APIKey store (#52781)
* move apikey store into a separate service

* add apikey service to wire graph

* fix linter

* switch api to use apikey service

* fix provideservice in tests

* add apikey service test double

* try different sql syntax

* rolling back the dialect

* trigger drone

* trigger drone
2022-08-02 16:55:19 +02:00
28e27e1365 Encryption: De-duplicate encryption code with extensible service (#52472)
* Encryption: De-duplicate encryption code with extensible service

* Fix Wire injections

* Fix tests

* Register reload handler
2022-08-02 15:08:09 +02:00
085ae014cd Sync: Move ActiveTokenCount to a new service (#52991)
* Move ActiveTokenCount to a new service

* Fixing tests

* fix `RootSystem`

Co-authored-by: Artur Wierzbicki <artur.wierzbicki@grafana.com>
2022-07-29 16:30:46 +02:00
197acd73c0 Storage: support git + github backed roots (#52192) 2022-07-28 23:26:44 -07:00
7ba076de10 Chore: Move swagger definitions to the handlers (#52643) 2022-07-27 09:54:37 -04:00
5ce4baf6f5 Correlations: Add CreateCorrelation HTTP API (#51630)
* Correlations: add migration

* Correlations: Add CreateCorrelation API

* Correlations: Make correlations work with provisioning

* Handle version changes

* Fix lining error

* lint fixes

* rebuild betterer results

* add a UID to each correlation

* Fix lint errors

* add docs

* better wording in API docs

* remove leftover comment

* handle ds updates

* Fix error message typo

* add bad data test

* make correlations a separate table

* skip readonly check when provisioning correlations

* delete stale correlations when datasources are deleted

* restore provisioned readonly ds

* publish deletion event with full data

* generate swagger and HTTP API docs

* apply source datasource permission to create correlation API

* Fix tests & lint errors

* ignore empty deletion events

* fix last lint errors

* fix more lint error

* Only publish deletion event if datasource was actually deleted

* delete DS provisioning deletes correlations, added & fixed tests

* Fix unmarshalling tests

* Fix linting errors

* Fix deltion event tests

* fix small linting error

* fix lint errors

* update betterer

* fix test

* make path singular

* Revert "make path singular"

This reverts commit 420c3d315e83ef9a7154766c812a24c7b02fc104.

* add integration tests

* remove unneeded id from correlations table

* update spec

* update leftover references to CorrelationDTO

* fix tests

* cleanup tests

* fix lint error
2022-07-25 15:19:07 +01:00
fb24ff3081 Auth: Add prometheus metrics for case insensitive ids (#52162)
* rename file

* add promstats for authinfostore every 4h

* add mixed cased users

* cherry pick and cleanup

* fix: query login,email instead of count()

* FIX: mysql missing alias to subquery

* fix: query working on mysql, sqllite3, and pg
2022-07-21 18:11:35 +01:00
e1785f4eb4 Secret migration from Sql KV Store to Secret Plugin (#52191)
* Created PluginSecretMigrationService to be able to migrate from the secrets table from the database to the secret plugin. Added migration which takes all the secrets at the sql store and stores it in the plugin. Then deletes all the secrets from the sql
* Added secretsKVStoreSQL.GetAll() method to return all the secrets at the sql table
* Renaming kvstore_test.go as sql_test.go, adding GetAll test case. Fixing decryption of keys
2022-07-19 14:42:23 -03:00
c061b66d5f Chore: Split delete user method (#52216)
* Remove user from preferences, stars, orguser, team member

* Fix lint

* Add Delete user from org and dashboard acl

* Delete user from user auth

* Add DeleteUser to quota

* Add test files and adjust user auth store

* Rename package in wire for user auth

* Import Quota Service interface in other services

* do the same in tests

* fix lint tests

* Fix tests

* Add some tests

* Rename InsertUser and DeleteUser to InsertOrgUser and DeleteOrgUser

* Rename DeleteUser to DeleteByUser in quota

* changing a method name in few additional places

* Fix in other places

* Fix lint

* Fix tests

* Chore: Split Delete User method

* Add fakes for userauth

* Add mock for access control Delete User permossion, use interface

* Use interface for ream guardian

* Add simple fake for dashboard acl

* Add go routines, clean up, use interfaces

* fix lint

* Update pkg/services/user/userimpl/user_test.go

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* Update pkg/services/user/userimpl/user_test.go

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* Update pkg/services/user/userimpl/user_test.go

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* Add wrapper for not service account error

* fix indentation

* Use fmt for error wrapper

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
2022-07-19 16:01:05 +02:00
fb379ae436 Chore: Introduce playlist service (#52252)
* Store: Introduce playlist service

* Integrate playlist service

* Update swagger
2022-07-18 05:26:35 -04:00
17ec9cac83 Add delete user from other services/stores (#51912)
* Remove user from preferences, stars, orguser, team member

* Fix lint

* Add Delete user from org and dashboard acl

* Delete user from user auth

* Add DeleteUser to quota

* Add test files and adjust user auth store

* Rename package in wire for user auth

* Import Quota Service interface in other services

* do the same in tests

* fix lint tests

* Fix tests

* Add some tests

* Rename InsertUser and DeleteUser to InsertOrgUser and DeleteOrgUser

* Rename DeleteUser to DeleteByUser in quota

* changing a method name in few additional places

* Fix in other places

* Fix lint

* Fix tests

* Rename DeleteOrgUser to DeleteUserFromAll

* Update pkg/services/org/orgimpl/org_test.go

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

* Update pkg/services/preference/prefimpl/inmemory_test.go

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

* Rename Acl to ACL

* Fix wire after merge with main

* Move test to uni test

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2022-07-15 18:06:44 +02:00
2d8a91a846 Secrets: Improve unified secrets migration and implement compatibility flag (#50463)
* Implement disableSecretsCompatibility flag

* Allow secret deletion right after migration

* Use dialect.Quote for secure_json_data on secret deletion

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* Set secure_json_data to NULL instead of empty json

* Run toggles_gen_test and use generated flag variable

* Add ID to delete data source secrets command on function call

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* Remove extra query to get datasource on secret deletion

* Fix linting issues with CHANGELOG.md

* Use empty json string when deleting secure json data

* Implement secret migration as a background process

* Refactor secret migration as a background service

* Refactor migration to be inside secret store

* Re-add secret deletion function removed on merge

* Try using transaction to fix db lock during tests

* Disable migration for pipeline debugging

* Try adding sleep to fix database lock

* Remove unecessary time sleep from migration

* Fix merge issue, replace models with datasources

* Try event listener approach

* Fix merge issue, replace models with datasources

* Fix linting issues with unchecked error

* Remove unecessary trainling new line

* Increase wait interval on background secret migration

* Rename secret store migration folder for consistency

* Convert background migration to blocking

* Fix number of arguments on server tests

* Check error value of secret migration provider

* Fix linting issue with method varaible

* Revert unintended change on background services

* Move secret migration service provider to wire.go

* Remove unecessary else from datasource service

* Move transaction inside loop on secret migration

* Remove unecessary GetServices function

* Remove unecessary interface after method removal

* Rename Run to Migrate on secret migration interface

* Rename secret migrations service variable on server

* Use MustBool on datasource secret migration

* Revert changes to GetDataSources

* Implement GetAllDataSources function

* Remove DeleteDataSourceSecrets function

* Move datasource secret migration to datasource service

* Remove unecessary properties from datasource secret migration

* Make DecryptLegacySecrets a private method

* Remove context canceled check on secret migrator

* Log error when fail to unmarshal datasource secret

* Add necessary fields to update command on migration

* Handle high availability on secret migration

* Use kvstore for datasource secret migration status

* Add error check for migration status set on kvstore

* Remove NewSecretMigrationService from server tests

* Use const for strings on datasource secrets migration

* Test all cases for datasources secret migrations

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
2022-07-12 17:27:37 -03:00
d85df0a560 Service Accounts: Managed permissions for service accounts (#51818)
* backend changes

* frontend changes

* linting

* nit

* import order

* allow SA creator to access the SA page

* fix merge

* tests

* fix frontend tests

Co-authored-by: alexanderzobnin alexanderzobnin@gmail.com
2022-07-08 05:53:18 -04:00
db9f80a7a3 ServiceAccounts: Add Prometheus metrics service (#51831)
* ServiceAccounts: Add Prometheus metrics service

* add missing init in tests
2022-07-07 10:03:16 -04:00
e96f67ae2e Renderer: Add sanitize API (#50936)
* svg fun

* #50597: add proto

* #50597: add sanitizer methods

* #50597: add provider

* #50597: use sanitizer

* #50597: use sanitizer

* update grafana to match new api

* add comments

* add capability check

* add timing

* update sanitize path

* improve log message

* strings.HasPrefix rather than filepath.IsAbs

* filepath.Clean + filepath.ToSlash for windows

* read 404

* remove `path.clean` from `getPathAndScope`

* add resp body close

* remove unneeded prop

* Update pkg/services/rendering/rendering.go

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>

* remove test files

* filepath.ToSlash correct wrapping

* filepath.ToSlash correct wrapping

* filepath.ToSlash comment

* compilation error

* lint fix

* fix error message

* Update pkg/services/rendering/rendering.go

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>

* add `image/svg+xml` mime type

* refactored log

* refactored log

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>
2022-07-07 15:32:18 +04:00
eacee08135 public dashboards: move into into its own service (#51358)
This PR moves public dashboards into its own self contained service including API, Service, Database, and Models. Routes are mounted on the Grafana HTTPServer by the API service at injection time with wire.go. The main  route that loads the frontend for public dashboards is still handled by the API package.

Co-authored-by: Jesse Weaver <jesse.weaver@grafana.com>
Co-authored-by: Owen Smallwood <owen.smallwood@grafana.com>
2022-07-06 17:51:44 -06:00