100 Commits

Author SHA1 Message Date
e7b800f35a Rendering: Add support for rate limiter (#103987)
* Rendering: Add support for rate limiter

* remove unnecessary golint exception

* add lint exception back
2025-04-24 15:31:19 +02:00
d7c554c25e Renderer: Fix regression on callback URL in plugin mode (#103787)
* Renderer: Fix regression on callback URL in plugin mode

* cleanup

* refactor to not mutate cfg object

* apply review feedback
2025-04-11 15:34:10 +02:00
757be6365a CI: Bump golangci-lint to 2.0.2 (#103572) 2025-04-10 14:42:23 +02:00
5996e102af Image Render: Propagate traces in HTTP mode (#100559)
* Rendering: Propagate traces in HTTP mode

* update ini file

* update docs

* update doc

* fix imports

* fix go files

* fix CI

* update ini config

* remove servicename

---------

Co-authored-by: AgnesToulet <35176601+AgnesToulet@users.noreply.github.com>
Co-authored-by: nmarrs <nathanielmarrs@gmail.com>
2025-03-06 12:02:00 -05:00
21bfdd445f Image Renderer: Minor refactor cleanup (#100089) 2025-02-06 08:01:11 -07:00
6eac07c3a7 Image Renderer: Add support for SSL in plugin mode (#98009) 2025-02-04 10:44:02 -08:00
4ae627e420 Render Service: Add trace ID to logs (#98006)
* Rendering: add logs

* fix tests and lint issues

* add context to logs

* update timed out logs to error logs
2025-01-09 10:19:11 +01:00
1c4d30f73c Rendering: Add logs (#97885)
* Rendering: add logs

* fix tests and lint issues
2024-12-13 09:42:35 +01:00
99d8025829 Chore: Move identity and errutil to apimachinery module (#89116) 2024-06-13 07:11:35 +03:00
42126398be Rendering: add userID in rendering log (#88132) 2024-05-23 09:41:53 +02:00
c9c6445554 Chore: Refactor render via http (#84613) 2024-05-14 07:24:18 -03:00
5687243d0b Feature Flags: use FeatureToggles interface where possible (#85131)
* Feature Flags: use FeatureToggles interface where possible

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* Replace TestFeatureToggles with existing WithFeatures

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

---------

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
2024-04-04 12:22:31 -04:00
58170d4141 Rendering: Add PDFRendering capability (#84438)
* Rendering: Add PDFRendering capability

* fix tests
2024-03-19 08:48:46 +01:00
28e66b4ad8 Rendering: Adds PDF support behind feature toggle (#81811)
* start pdf refactor

* Update AppChrome.tsx

* Update AppChrome.tsx

* add encoding param to rendering grpc service

* fix plugin mode

* clean up

* fix backend tests

* fix lint errors

* Support pdf encoding in render http api

---------

Co-authored-by: Torkel Ödegaard <torkel@grafana.com>
2024-02-08 13:09:34 +01:00
6768c6c059 Chore: Remove public vars in setting package (#81018)
Removes the public variable setting.SecretKey plus some other ones. 
Introduces some new functions for creating setting.Cfg.
2024-01-23 12:36:22 +01:00
1cec6195f1 Rendering: Fix plugin initialization (#80013) 2024-01-04 11:30:01 +01:00
ce8fd14f1f Plugins: Make renderer service load renderer plugin (#77854)
* rendering service loads renderer plugin

* update naming

* tidy

* apply PR feedback

* fix missing feature manager

* fix step

* set plugin
2023-12-14 17:33:29 +01:00
f69fd3726b FeatureToggles: Add context and and an explicit global check (#78081) 2023-11-14 12:50:27 -08:00
025b2f3011 Chore: use any rather than interface{} (#74066) 2023-08-30 18:46:47 +03:00
878e94ae25 Rendering: Use image for server timeout errors (#73462) 2023-08-18 11:50:49 +03:00
87a0c95164 Rendering: Experimental support to use JWTs as auth method (#60841)
* Rendering: Add support for auth through JWT

* Goimports

* Apply review suggestions

* Correct feature toggle ref

* Minor changes
2023-04-03 18:53:38 +02:00
eb507dca89 Remotecache: rename setbytearray/getbytearray to set/get and remove codec (#64470)
Signed-off-by: bergquist <carl.bergquist@gmail.com>
2023-03-10 13:57:29 +01:00
b88206d98f Cache: Refactor cache clients to use byte array (#62930)
Signed-off-by: bergquist <carl.bergquist@gmail.com>
2023-02-08 10:30:20 +01:00
bc2813ef06 Chore: Fix goimports grouping in pkg/services (#62420)
* fix goimports

* fix goimports order
2023-01-30 08:21:27 +00:00
48b33ab521 Plugins: Unsigned chromium file should not invalidate signature for Renderer plugin (#59104)
* Plugins: Unsigned chromium file should not invalidate signature for Renderer plugin

* fix test

* re-work solution
2022-12-19 12:46:27 +01:00
e9dc7fb85c Rendering: Add configuration options for renderKey lifetime (#57339)
* Add configuration options for `renderKey` lifetime

* Rename config key to `render_key_lifetime`

* Update conf/defaults.ini

Co-authored-by: Joan López de la Franca Beltran <5459617+joanlopez@users.noreply.github.com>

* Add `render_key_lifetime` to sample.ini

Co-authored-by: Joan López de la Franca Beltran <5459617+joanlopez@users.noreply.github.com>
2022-11-03 12:06:55 +01:00
bff9671b81 Rendering: Check timeout error when the request failed (#57654)
* Check timeout error when the request failed

* Update error and check

* Uses errors.As to fix the lint
2022-10-27 17:27:03 +02:00
b287047052 Chore: Upgrade Go to 1.19.1 (#54902)
* WIP

* Set public_suffix to a pre Ruby 2.6 version

* we don't need to install python

* Stretch->Buster

* Bump versions in lib.star

* Manually update linter

Sort of messy, but the .mod-file need to contain all dependencies that
use 1.16+ features, otherwise they're assumed to be compiled with
-lang=go1.16 and cannot access generics et al.

Bingo doesn't seem to understand that, but it's possible to manually
update things to get Bingo happy.

* undo reformatting

* Various lint improvements

* More from the linter

* goimports -w ./pkg/

* Disable gocritic

* Add/modify linter exceptions

* lint + flatten nested list

Go 1.19 doesn't support nested lists, and there wasn't an obvious workaround.
https://go.dev/doc/comment#lists
2022-09-12 12:03:49 +02:00
ecdcafb258 manager exposes renderer + secrets manager (#54629) 2022-09-02 14:20:10 +02:00
6ec06f66b9 Rendering: Add support for renderer token (#54425)
(cherry picked from commit a4f75cc0438712c90b02d24740416f8615e3a0cb)
2022-08-30 12:09:38 +02:00
6afad51761 Move SignedInUser to user service and RoleType and Roles to org (#53445)
* Move SignedInUser to user service and RoleType and Roles to org

* Use go naming convention for roles

* Fix some imports and leftovers

* Fix ldap debug test

* Fix lint

* Fix lint 2

* Fix lint 3

* Fix type and not needed conversion

* Clean up messages in api tests

* Clean up api tests 2
2022-08-10 11:56:48 +02:00
9afe845d5c Sanitization: fix log message (#52401) 2022-07-18 06:46:57 -07:00
e96f67ae2e Renderer: Add sanitize API (#50936)
* svg fun

* #50597: add proto

* #50597: add sanitizer methods

* #50597: add provider

* #50597: use sanitizer

* #50597: use sanitizer

* update grafana to match new api

* add comments

* add capability check

* add timing

* update sanitize path

* improve log message

* strings.HasPrefix rather than filepath.IsAbs

* filepath.Clean + filepath.ToSlash for windows

* read 404

* remove `path.clean` from `getPathAndScope`

* add resp body close

* remove unneeded prop

* Update pkg/services/rendering/rendering.go

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>

* remove test files

* filepath.ToSlash correct wrapping

* filepath.ToSlash correct wrapping

* filepath.ToSlash comment

* compilation error

* lint fix

* fix error message

* Update pkg/services/rendering/rendering.go

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>

* add `image/svg+xml` mime type

* refactored log

* refactored log

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>
2022-07-07 15:32:18 +04:00
687e79538b Alerting: Add a general screenshot service and alerting-specific image service. (#49293)
This commit adds a pkg/services/screenshot package for taking and uploading screenshots of Grafana dashboards. It supports taking screenshots of both dashboards and individual panels within a dashboard, using the rendering service.

The screenshot package has the following services, most of which can be composed:

BrowserScreenshotService (Takes screenshots with headless Chrome)
CachableScreenshotService (Caches screenshots taken with another service such as BrowserScreenshotService)
NoopScreenshotService (A no-op screenshot service for tests)
SingleFlightScreenshotService (Prevents duplicate screenshots when taking screenshots of the same dashboard or panel in parallel)
ScreenshotUnavailableService (A screenshot service that returns ErrScreenshotsUnavailable)
UploadingScreenshotService (A screenshot service that uploads taken screenshots)

The screenshot package does not support wire dependency injection yet. ngalert constructs its own version of the service. See https://github.com/grafana/grafana/issues/49296

This PR also adds an ImageScreenshotService to ngAlert. This is used to take screenshots with a screenshotservice and then store their location reference for use by alert instances and notifiers.
2022-05-22 22:33:49 +08:00
897db011eb Add error options for rendering to return errors on failure (#48864) 2022-05-09 19:11:24 +01:00
a4381ebc91 Previews: create crawler auth setup service (#47349)
* #46968: add `RetrieveServiceAccountIdByName` to serviceaccounts service

* #46968: improve error logging in rendering service

* #46968: add oss crawler account setup

* #46968: fix tests

* #46968: switch back to ROLE_ADMIN

* #46968: rename to crawlerAuth

* comment crawler_auth.go
2022-04-12 19:34:04 +02:00
7643ae6c5e Rendering: periodically refresh remote plugin version (#45505)
* #44449: refresh image renderer version on a schedule

* #44449: simplify
2022-02-18 20:25:01 +04:00
a025109647 Dash previews: populate crawler queue from SQL query (#44083)
* add SQL migrations

* dashboard previews from sql: poc

* added todos

* refactor: use the same enums where possible

* use useEffect, always return json

* added todo

* refactor + delete files after use

* refactor + fix manual thumbnail upload

* refactor: move all interactions with sqlStore to thumbnail repo

* refactor: remove file operations in thumb crawler/service

* refactor: fix dashboard_thumbs sql store

* refactor: extracted thumbnail fetching/updating to a hook

* refactor: store thumbnails in redux store

* refactor: store thumbnails in redux store

* refactor: private'd repo methods

* removed redux storage, saving images as blobs

* allow for configurable rendering timeouts

* added 1) query for dashboards with stale thumbnails, 2) command for marking thumbnails as stale

* use sql-based queue in crawler

* ui for marking thumbnails as stale

* replaced `stale` boolean prop with `state` enum

* introduce rendering session

* compilation errors

* fix crawler stop button

* rename thumbnail state frozen to locked

* #44449: fix merge conflicts

* #44449: remove thumb methods from `Store` interface

* #44449: clean filepath, defer file closing

* #44449: fix rendering.Theme cyclic import

* #44449: linting

* #44449: linting

* #44449: mutex'd crawlerStatus access

* #44449: added integration tests for `sqlstore.dashboard_thumbs`

* #44449: added comments to explain the `ThumbnailState` enum

* #44449: use os.ReadFile rather then os.Open

* #44449: always enable dashboardPreviews feature during integration tests

* #44449: remove sleep time, adjust number of threads

* #44449: review fix: add `orgId` to `DashboardThumbnailMeta`

* #44449: review fix: automatic parsing of thumbnailState

* #44449: lint fixes

* #44449: review fix: prefer `WithDbSession` over `WithTransactionalDbSession`

* #44449: review fix: add a comment explaining source of the filepath

* #44449: review fix: added filepath validation

* #44449: review fixes https://github.com/grafana/grafana/pull/45063/files @fzambia

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
2022-02-09 13:23:32 +04:00
254c59725e Rendering: add capabilities check (#44470)
* #44449: add feature check to rendering service

* #44449: formatting

* #44449: rename feature -> capability (https://developer.mozilla.org/en-US/docs/Web/API/MediaStreamTrack/getCapabilities, https://developer.mozilla.org/en-US/docs/Web/API/InputDeviceInfo/getCapabilities)

* #44449: refactor

* #44449: remove commented code

* Update pkg/services/rendering/capabilities.go

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>

* #44449: review fixes

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>
2022-01-28 21:24:15 +04:00
5148250366 Rendering service - add optional RenderingSession (#44098)
* rendering service changes:
- make node-renderer request timeout configurable
- introduce optional RenderingSession providing a long-lived session key

* remove console logs

* added comment explaining empty "afterRequest" method

* fix compilation error

* update imports formatting

* Update pkg/services/rendering/interface.go

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>

* Update pkg/services/rendering/rendering.go

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>

* review fix: extract renderKey related functions/structs to auth.go

* #44449: private'd `rendering.getRequestTimeout`

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>
2022-01-27 02:02:19 +04:00
b404aae9c3 Crawler: use existing render service to generate dashboard thumbnails (#43515)
Co-authored-by: Artur Wierzbicki <artur@arturwierzbicki.com>
2022-01-10 08:21:35 -08:00
b8852ef6a3 Chore: Remove context.TODO() (#43409)
* Remove context.TODO() from services

* Fix live test

* Remove context.TODO
2021-12-22 11:02:42 +01:00
be578e5700 Improve handling of old/slow-to-start remote renderer (#40492)
* Assume the remote renderer is old if it returns 404 to the version endpoint

* Retry fetch of remote image renderer version on failure

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>
2021-12-01 08:58:43 -07:00
9f7942099b Rendering: Add light theme for errors (#41616)
* Add light theme for error rendering

* Fix lint

* Update images and add rendering_limit_x images
2021-11-17 12:18:47 +01:00
b80fbe03f0 Plugins: Refactor Plugin Management (#40477)
* add core plugin flow

* add instrumentation

* move func

* remove cruft

* support external backend plugins

* refactor + clean up

* remove comments

* refactor loader

* simplify core plugin path arg

* cleanup loggers

* move signature validator to plugins package

* fix sig packaging

* cleanup plugin model

* remove unnecessary plugin field

* add start+stop for pm

* fix failures

* add decommissioned state

* export fields just to get things flowing

* fix comments

* set static routes

* make image loading idempotent

* merge with backend plugin manager

* re-use funcs

* reorder imports + remove unnecessary interface

* add some TODOs + remove unused func

* remove unused instrumentation func

* simplify client usage

* remove import alias

* re-use backendplugin.Plugin interface

* re order funcs

* improve var name

* fix log statements

* refactor data model

* add logic for dupe check during loading

* cleanup state setting

* refactor loader

* cleanup manager interface

* add rendering flow

* refactor loading + init

* add renderer support

* fix renderer plugin

* reformat imports

* track errors

* fix plugin signature inheritance

* name param in interface

* update func comment

* fix func arg name

* introduce class concept

* remove func

* fix external plugin check

* apply changes from pm-experiment

* fix core plugins

* fix imports

* rename interface

* comment API interface

* add support for testdata plugin

* enable alerting + use correct core plugin contracts

* slim manager API

* fix param name

* fix filter

* support static routes

* fix rendering

* tidy rendering

* get tests compiling

* fix install+uninstall

* start finder test

* add finder test coverage

* start loader tests

* add test for core plugins

* load core + bundled test

* add test for nested plugin loading

* add test files

* clean interface + fix registering some core plugins

* refactoring

* reformat and create sub packages

* simplify core plugin init

* fix ctx cancel scenario

* migrate initializer

* remove Init() funcs

* add test starter

* new logger

* flesh out initializer tests

* refactoring

* remove unused svc

* refactor rendering flow

* fixup loader tests

* add enabled helper func

* fix logger name

* fix data fetchers

* fix case where plugin dir doesn't exist

* improve coverage + move dupe checking to loader

* remove noisy debug logs

* register core plugins automagically

* add support for renderer in catalog

* make private func + fix req validation

* use interface

* re-add check for renderer in catalog

* tidy up from moving to auto reg core plugins

* core plugin registrar

* guards

* copy over core plugins for test infra

* all tests green

* renames

* propagate new interfaces

* kill old manager

* get compiling

* tidy up

* update naming

* refactor manager test + cleanup

* add more cases to finder test

* migrate validator to field

* more coverage

* refactor dupe checking

* add test for plugin class

* add coverage for initializer

* split out rendering

* move

* fixup tests

* fix uss test

* fix frontend settings

* fix grafanads test

* add check when checking sig errors

* fix enabled map

* fixup

* allow manual setup of CM

* rename to cloud-monitoring

* remove TODO

* add installer interface for testing

* loader interface returns

* tests passing

* refactor + add more coverage

* support 'stackdriver'

* fix frontend settings loading

* improve naming based on package name

* small tidy

* refactor test

* fix renderer start

* make cloud-monitoring plugin ID clearer

* add plugin update test

* add integration tests

* don't break all if sig can't be calculated

* add root URL check test

* add more signature verification tests

* update DTO name

* update enabled plugins comment

* update comments

* fix linter

* revert fe naming change

* fix errors endpoint

* reset error code field name

* re-order test to help verify

* assert -> require

* pm check

* add missing entry + re-order

* re-check

* dump icon log

* verify manager contents first

* reformat

* apply PR feedback

* apply style changes

* fix one vs all loading err

* improve log output

* only start when no signature error

* move log

* rework plugin update check

* fix test

* fix multi loading from cfg.PluginSettings

* improve log output #2

* add error abstraction to capture errors without registering a plugin

* add debug log

* add unsigned warning

* e2e test attempt

* fix logger

* set home path

* prevent panic

* alternate

* ugh.. fix home path

* return renderer even if not started

* make renderer plugin managed

* add fallback renderer icon, update renderer badge + prevent changes when renderer is installed

* fix icon loading

* rollback renderer changes

* use correct field

* remove unneccessary block

* remove newline

* remove unused func

* fix bundled plugins base + module fields

* remove unused field since refactor

* add authorizer abstraction

* loader only returns plugins expected to run

* fix multi log output
2021-11-01 10:53:33 +01:00
37cbed4b48 Rendering: Check if default image exists to avoid breaking reports if it's moved (#40930) 2021-10-27 16:08:03 +02:00
3df001db93 ImageRendering: fix leakage of the rendering inflight counter (#34607) 2021-08-28 09:05:03 +02:00
78596a6756 Migrate to Wire for dependency injection (#32289)
Fixes #30144

Co-authored-by: dsotirakis <sotirakis.dim@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Leon Sorokin <leeoniya@gmail.com>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: spinillos <selenepinillos@gmail.com>
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
2021-08-25 15:11:22 +02:00
0757ba1c64 Renderer: delete debug.log file when Grafana is stopped (#37367) 2021-07-30 17:29:02 +02:00
5f6c172b5a Renderer: add version information (#34964)
* Renderer: add version information

* fix alerting test
2021-06-04 14:33:49 +03:00