10 Commits

Author SHA1 Message Date
18daa6754c Previews: datasource permissions (#52747)
* Previews: datasource permissions

* lint

* simplify - force non-null `ds_uids`

* add `canBeDisabled` to search service

* add `IncludeThumbnailsWithEmptyDsUids`

* remove force refresh migration

* refactor main preview service

* add safeguard

* revert ticker interval

* update testdata

* fix test

* add mock search service

* add datasources lookup test

* update migration

* extract ds lookup to its own package to avoid cyclic imports

* lint

* fix dashbaord extract, use the real datasource lookup in tests. IS IT BULLETPROOF YET?!

* fix dashbaord extract, use the real datasource lookup in tests. IS IT BULLETPROOF YET?!

* remove stale log

* consistent casing

* pass context to `createServiceAccount`

* filter out the special grafana ds
2022-07-28 16:40:26 +04:00
9de00c8eb2 chore/backend: move dashboard errors to dashboard service (#51593)
* chore/backend: move dashboard errors to dashboard service

Dashboard-related models are slowly moving out of the models package and into dashboard services. This commit moves dashboard-related errors; the rest will come in later commits.

There are no logical code changes, this is only a structural (package) move.

* lint lint lint
2022-06-30 09:31:54 -04:00
bd35e6917a Chore: Exclude integration tests from running on test-backend step (#50359)
* Chore: Exclude integration tests from running on test-backend step

* Remove -v from go test command

* Add check to skip integration tests before each integration test

* Try to restart pipeline

* Retrying to make pipeline run
2022-06-10 11:46:21 -04:00
e9f8d582c8 Chore: Remove dashboard version from models (#50287)
* Remove dashbpard version from models

* Fix lint

* Fix api & sqlstore tests

* Remove integration tags

* Fix lint again

* Add integration test to correct namespace

* Lont fix 2

* Change Id to ID in dashVersionMeta
2022-06-08 12:22:55 +02:00
bdf50f3dd2 Dashboards: Split GetDashboardVersions method (#49967)
* Split GetDashboarVersions method

* Add sqlstore dialect and tests

* Fix signature of PAtchPreference

* Add GetDialect to sqlstore and remove GetDashboardVersions

* Add GetDialect to db interface

* Implement List

* add deleted test function

* Remove GetDialect from sqlstore interface

* Remove deleted method from mock

* Refactor test
2022-06-02 15:59:05 +02:00
c63ebc887b Chore: Run integration tests without grabpl (#49448)
* Chore: Run integration tests without grabpl

* Add new step for integration tests in lib.star

* Remove old integration test step from lib.star

* Change drone signature

* Fix: Edit starlark integration step to not affect enterprise

* Remove all build tags & rename starlark integration test step

* Resync .drone.yml with .drone.star

* Fix lint errors

* Fix lint errors

* Fix lint errors

* Fix more lint errors

* Fix another lint error

* Rename integration test step

* Fix last lint error

* Recomment enterprise step

* Remove comment from Makefile

Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
2022-06-01 14:55:22 -04:00
50c2b4682a Chore: Rename integration tests (#49438)
* Chore: Rename integration tests

* Remove one Integration

Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
2022-05-24 11:04:03 +02:00
6c76aa71e8 Previews: capability check (#44601)
* add SQL migrations

* dashboard previews from sql: poc

* added todos

* refactor: use the same enums where possible

* use useEffect, always return json

* added todo

* refactor + delete files after use

* refactor + fix manual thumbnail upload

* refactor: move all interactions with sqlStore to thumbnail repo

* refactor: remove file operations in thumb crawler/service

* refactor: fix dashboard_thumbs sql store

* refactor: extracted thumbnail fetching/updating to a hook

* refactor: store thumbnails in redux store

* refactor: store thumbnails in redux store

* refactor: private'd repo methods

* removed redux storage, saving images as blobs

* allow for configurable rendering timeouts

* added 1) query for dashboards with stale thumbnails, 2) command for marking thumbnails as stale

* use sql-based queue in crawler

* ui for marking thumbnails as stale

* replaced `stale` boolean prop with `state` enum

* introduce rendering session

* compilation errors

* fix crawler stop button

* rename thumbnail state frozen to locked

* #44449: fix merge conflicts

* #44449: remove thumb methods from `Store` interface

* #44449: clean filepath, defer file closing

* #44449: fix rendering.Theme cyclic import

* #44449: linting

* #44449: linting

* #44449: mutex'd crawlerStatus access

* #44449: added integration tests for `sqlstore.dashboard_thumbs`

* #44449: added comments to explain the `ThumbnailState` enum

* #44449: use os.ReadFile rather then os.Open

* #44449: always enable dashboardPreviews feature during integration tests

* #44449: add /previews/system-requirements API

* #44449: remove sleep time, adjust number of threads

* #44449: review fix: add `orgId` to `DashboardThumbnailMeta`

* #44449: review fix: automatic parsing of thumbnailState

* #44449: update returned json

* #44449: UI changes - dashboard previews sytem req check

* #44449: lint fixes

* #44449: fix tests

* #44449: typo

* #44449: fix getSystemRequirements API: return 200 even if we plugin version is invalid

* #44449: fix getSystemRequirements API: don't return SemverConstraint on error

* #44449: fix getSystemRequirements API

* #44449: fix previews sytem requirements text

* #44449: add `doThumbnailsExist` to repo

* #44449: remove redux api

* #44449: add missing model

* #44449: implement frontedsettings-driven capability check

* #44449: simplify

* #44449: revert test changes

* #44449: add dummy setup settings

* #44449: implicit typing over `FC<Props>`

* #44449: refactor conditionals

* #44449: replace `getText` with a react component

* #44449: fix component interface

* #44449: add onRemove to `PreviewsSystemRequirements` alert

* #44449: add bottom/top margin to previewSystemRequirements modal

* #44449: merge conflict fix

* #44449: remove console.log

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
2022-02-16 21:49:50 +04:00
0276b029fc Previews: crawler as a background service (#44891)
* add SQL migrations

* dashboard previews from sql: poc

* added todos

* refactor: use the same enums where possible

* use useEffect, always return json

* added todo

* refactor + delete files after use

* refactor + fix manual thumbnail upload

* refactor: move all interactions with sqlStore to thumbnail repo

* refactor: remove file operations in thumb crawler/service

* refactor: fix dashboard_thumbs sql store

* refactor: extracted thumbnail fetching/updating to a hook

* refactor: store thumbnails in redux store

* refactor: store thumbnails in redux store

* refactor: private'd repo methods

* removed redux storage, saving images as blobs

* allow for configurable rendering timeouts

* added 1) query for dashboards with stale thumbnails, 2) command for marking thumbnails as stale

* use sql-based queue in crawler

* ui for marking thumbnails as stale

* replaced `stale` boolean prop with `state` enum

* introduce rendering session

* compilation errors

* fix crawler stop button

* rename thumbnail state frozen to locked

* #44449: fix merge conflicts

* #44449: remove thumb methods from `Store` interface

* #44449: clean filepath, defer file closing

* #44449: fix rendering.Theme cyclic import

* #44449: linting

* #44449: linting

* #44449: mutex'd crawlerStatus access

* #44449: added integration tests for `sqlstore.dashboard_thumbs`

* #44449: added comments to explain the `ThumbnailState` enum

* #44449: use os.ReadFile rather then os.Open

* #44449: always enable dashboardPreviews feature during integration tests

* #44449: remove sleep time, adjust number of threads

* #44449: review fix: add `orgId` to `DashboardThumbnailMeta`

* #44449: review fix: automatic parsing of thumbnailState

* #44449: lint fixes

* #44449: crawler as a background service v0.1

* #44449: use ServerLockService

* #44449: use ServerLockService

* #44449: review fix: prefer `WithDbSession` over `WithTransactionalDbSession`

* #44449: review fix: add a comment explaining source of the filepath

* #44449: review fix: added filepath validation

* #44449: fix FindDashboardsWithStaleThumbnails to include `theme` and `kind` in search params

* #44449: fix FindDashboardsWithStaleThumbnails to include `theme` and `kind` in search params

* #44449: create function for crawler on demand

* #44449: improve crawler logging

* #44449: fix wire

* #44449: uncomment dummy thumb service, fix ticker interval

* #44449: prevent race condition

* #44449: improve logging

* #44449: fix theme

* #44449: review fixes https://github.com/grafana/grafana/pull/45063/files @fzambia

* #44449: add missing unlock

* #44449: merge

* #44449: review fix - logger @fzambia https://github.com/grafana/grafana/pull/45063/files

* #44449: formatting

* #44449: merge conflict fix

* #44449: merge conflict fix

* #44449: merge conflict fix

* #44449: naming fix

* #44449: update authOpts

* #44449: change authOpts.role back to admin

* #44449: fix `walk` signature, move ctx to a first argument

* #44449: add `dashboardPreviewsScheduler` feature flag

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
2022-02-10 22:45:00 +04:00
a025109647 Dash previews: populate crawler queue from SQL query (#44083)
* add SQL migrations

* dashboard previews from sql: poc

* added todos

* refactor: use the same enums where possible

* use useEffect, always return json

* added todo

* refactor + delete files after use

* refactor + fix manual thumbnail upload

* refactor: move all interactions with sqlStore to thumbnail repo

* refactor: remove file operations in thumb crawler/service

* refactor: fix dashboard_thumbs sql store

* refactor: extracted thumbnail fetching/updating to a hook

* refactor: store thumbnails in redux store

* refactor: store thumbnails in redux store

* refactor: private'd repo methods

* removed redux storage, saving images as blobs

* allow for configurable rendering timeouts

* added 1) query for dashboards with stale thumbnails, 2) command for marking thumbnails as stale

* use sql-based queue in crawler

* ui for marking thumbnails as stale

* replaced `stale` boolean prop with `state` enum

* introduce rendering session

* compilation errors

* fix crawler stop button

* rename thumbnail state frozen to locked

* #44449: fix merge conflicts

* #44449: remove thumb methods from `Store` interface

* #44449: clean filepath, defer file closing

* #44449: fix rendering.Theme cyclic import

* #44449: linting

* #44449: linting

* #44449: mutex'd crawlerStatus access

* #44449: added integration tests for `sqlstore.dashboard_thumbs`

* #44449: added comments to explain the `ThumbnailState` enum

* #44449: use os.ReadFile rather then os.Open

* #44449: always enable dashboardPreviews feature during integration tests

* #44449: remove sleep time, adjust number of threads

* #44449: review fix: add `orgId` to `DashboardThumbnailMeta`

* #44449: review fix: automatic parsing of thumbnailState

* #44449: lint fixes

* #44449: review fix: prefer `WithDbSession` over `WithTransactionalDbSession`

* #44449: review fix: add a comment explaining source of the filepath

* #44449: review fix: added filepath validation

* #44449: review fixes https://github.com/grafana/grafana/pull/45063/files @fzambia

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
2022-02-09 13:23:32 +04:00