119 Commits

Author SHA1 Message Date
b287047052 Chore: Upgrade Go to 1.19.1 (#54902)
* WIP

* Set public_suffix to a pre Ruby 2.6 version

* we don't need to install python

* Stretch->Buster

* Bump versions in lib.star

* Manually update linter

Sort of messy, but the .mod-file need to contain all dependencies that
use 1.16+ features, otherwise they're assumed to be compiled with
-lang=go1.16 and cannot access generics et al.

Bingo doesn't seem to understand that, but it's possible to manually
update things to get Bingo happy.

* undo reformatting

* Various lint improvements

* More from the linter

* goimports -w ./pkg/

* Disable gocritic

* Add/modify linter exceptions

* lint + flatten nested list

Go 1.19 doesn't support nested lists, and there wasn't an obvious workaround.
https://go.dev/doc/comment#lists
2022-09-12 12:03:49 +02:00
736d035c65 Chore: Add context util that allow to provide cause of cancellation (#53918) 2022-08-24 10:24:41 -04:00
ce7593686d Errors: Allow using Base as an error type (#53824) 2022-08-22 11:07:33 +02:00
Jo
ca72cd570e Remove ioutil.ReadDir from usage (#53550)
* add depguard rule for ioutil

* replace ioutil.ReadDir with os.ReadDir

* use legacy option in depguard supported in golangci-lint v1.40

* replace ioutil.ReadDir with os.ReadDir

* return error for file info
2022-08-11 07:21:12 -04:00
58a4122624 Fix: Comment out flaky tests for TestReadProm (#52243)
* comment out flaky test

* commented out unused functions

* using t.skip()

* disable linting
2022-07-14 17:42:13 +01:00
dd6d71ee4b HTTP: Add function for using new style errors with fallback (#51627) 2022-07-13 13:14:28 +02:00
4a00c7ebde Storage: Add basic file upload management (#50638) 2022-07-05 10:53:41 -07:00
f4f31b40fd HeatmapNG: consolidate frame types & fix color ranging (#51089)
* rename yZeroDisplay -> yMinDisplay
* remove heatmap-cells-sparse frame type
* parse x bucket size to millis
* take into account hideLE & hideGE filters to auto-range color scale
* extract cell value range scanning to heatmapData
2022-06-20 15:38:13 -05:00
fd63ed540f loki: fix bug in labels framing (#51015) 2022-06-17 12:50:10 +02:00
f1834163ec ShortURL: Use new Error type (#50859) 2022-06-15 15:11:36 +02:00
264c2a9d1e Errors: Introduce error type with Grafana specific metadata (#47504) 2022-06-14 10:50:11 +02:00
a88408bfd1 Fix Reporter GetDiffsForField (#50264) 2022-06-07 11:16:05 +08:00
31630edf0c Chore: Remove Wrapf (#50128)
* Chore: Remove Wrapf

* Remove all Wrapf refs

* Remove last Wrapf ref

* Fix lint errors

* Remove Wrap and Wrapf definitions

* Remove unnecessary colon
2022-06-06 22:30:31 +02:00
64d93498de Chore: Update grafana-plugin-sdk-go to v0.136.0 (#50127) 2022-06-03 08:25:52 -04:00
33d4850c90 Prometheus: Fix sort issue in wide frames (#49660) 2022-05-25 23:32:55 -04:00
1f85101787 Util: Improve performance of strings.SplitString (#49115)
Replaces the regexp with calls to strings.ReplaceAll and strings.Fields
for simplicity and improved performance.

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
2022-05-25 14:10:22 -07:00
94b9c524a8 Prometheus: Streaming JSON parser performance improvements (#48792) 2022-05-24 16:17:11 -04:00
97759c75f4 Prometheus: add support for new _experimental_ sparse histograms/heatmaps (#47801)
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2022-05-16 02:47:18 +02:00
87e8521591 Prometheus: Implement Streaming JSON Parser (#48477)
use `prometheusStreamingJSONParser` feature toggle to enable
2022-05-13 20:28:54 +02:00
3ffe447c80 Add new features to DiffReport and Diff (#48788)
* simplify String for Diff
* add IsAddOperation and IsDeleteOperation to Diff
* add method Paths to DiffReport
2022-05-06 11:06:00 -04:00
ec666f8785 Converter: Add result type to frame meta (#48769) 2022-05-05 16:57:24 +02:00
da74dba7c8 Loki: backend: use streaming JSON parser, try2 (#48752)
* converter: remove __name__ customization because Loki does not do that

Loki does not handle __name__ in a special way.
for Prometheus, the caller can implement the formatting
by themselves

* converter: change labels-formatting

the labels.String() method does not handle strange values well

* loki: backend: use streaming-json parser

* more idiomatic code

Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>

* simpler row-length check

* simpler code

* fixed converter/prom tests

Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
2022-05-05 13:09:01 +02:00
3cade2f669 Revert "Loki: backend: use streaming JSON parser (#47656)" (#48747)
This reverts commit 46b40b6e82417661078be5f1c4a9d802e440d9ec.
2022-05-05 12:02:53 +02:00
46b40b6e82 Loki: backend: use streaming JSON parser (#47656)
* converter: remove __name__ customization because Loki does not do that

Loki does not handle __name__ in a special way.
for Prometheus, the caller can implement the formatting
by themselves

* converter: change labels-formatting

the labels.String() method does not handle strange values well

* loki: backend: use streaming-json parser

* more idiomatic code

Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>

* simpler row-length check

* simpler code

Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
2022-05-05 08:47:40 +02:00
9529c35efa Converter: Add support for parsing prometheus string (#48727) 2022-05-04 20:26:32 -04:00
35300a816a Prometheus: Add support for streaming scalar parsing (#48725) 2022-05-04 16:03:48 -04:00
6e6f6e3cce Converter: Add support for parsing error & warning from prometheus results (#48721) 2022-05-04 12:01:18 -07:00
9047058c43 Auth Proxy: non-ASCII headers encoding tests (#47110) 2022-05-02 13:08:20 +02:00
0afc542998 Chore: Try to fix flaky reverse proxy test (#47957) 2022-04-20 16:24:06 +02:00
4bc582570e Instrumentation: Proxy status code correction and various improvements (#47473)
For a proxied request, e.g. Grafana's datasource or plugin proxy:
If the request is cancelled, e.g. from the browser, the HTTP status code is 
now 499 Client closed request instead of 502 Bad gateway.
If the request times out, e.g. takes longer time than allowed, the HTTP status 
code is now 504 Gateway timeout instead of 502 Bad gateway.
This also means that request metrics and logs will get their status codes 
adjusted according to above.

Fixes #46337
Fixes #46338
2022-04-11 13:17:08 +02:00
110d4661d7 Converter: use streaming JSON parser to construct frames from loki/prometheus responses (#44520) 2022-04-04 17:22:14 -07:00
1dca39fb91 Auth Proxy: encoding of non-ASCII headers (#44797)
* Decode auth proxy headers using URL encoding

* Header encoding configuration via settings file

* Rename configuration setting to headers_encoded

* Quoted-printable encoding

* Tests for AuthProxy

* Fix encoding name

* Remove authproxy init
2022-03-04 04:58:27 -05:00
4502e40ed8 Alerting: Revert Revert "Alerting: Calculate diff for two AlertRules" (#46034)
* Revert "Revert "Alerting: Calculate diff for two AlertRules (#45877)" (#46023)"

This reverts commit 82aa5acba6b857d4eb7c6b5faf485ae6d20f7328.

* remove flakiness
2022-03-01 11:10:29 -05:00
82aa5acba6 Revert "Alerting: Calculate diff for two AlertRules (#45877)" (#46023)
This reverts commit 4e19d7df6352b0dcbb680aeee00cebc97a90d937.
2022-03-01 13:40:47 +01:00
4e19d7df63 Alerting: Calculate diff for two AlertRules (#45877)
* add custom diff reporter DiffReporter that reports only paths that have a difference
* create Diff method for AlertRule that returns DiffReport, which is an alias for []Diff

Tests:
* create copy method for AlertRule in testing
* create GenerateAlertQuery method in testing
2022-02-28 17:13:53 +01:00
605d056136 Security: Sync security changes on main (#45083)
* * Teams: Appropriately apply user id filter in /api/teams/:id and /api/teams/search
* Teams: Ensure that users searching for teams are only able see teams they have access to
* Teams: Require teamGuardian admin privileges to list team members
* Teams: Prevent org viewers from administering teams
* Teams: Add org_id condition to team count query
* Teams: clarify permission requirements in teams api docs
* Teams: expand scenarios for team search tests
* Teams: mock teamGuardian in tests

Co-authored-by: Dan Cech <dcech@grafana.com>

* remove duplicate WHERE statement

* Fix for CVE-2022-21702

(cherry picked from commit 202d7c190082c094bc1dc13f7fe9464746c37f9e)

* Lint and test fixes

(cherry picked from commit 3e6b67d5504abf4a1d7b8d621f04d062c048e981)

* check content type properly

(cherry picked from commit 70b4458892bf2f776302720c10d24c9ff34edd98)

* basic csrf origin check

(cherry picked from commit 3adaa5ff39832364f6390881fb5b42ad47df92e1)

* compare origin to host

(cherry picked from commit 5443892699e8ed42836bb2b9a44744ff3e970f42)

* simplify url parsing

(cherry picked from commit b2ffbc9513fed75468628370a48b929d30af2b1d)

* check csrf for GET requests, only compare origin

(cherry picked from commit 8b81dc12d8f8a1f07852809c5b4d44f0f0b1d709)

* parse content type properly

(cherry picked from commit 16f76f4902e6f2188bea9606c68b551af186bdc0)

* mentioned get in the comment

(cherry picked from commit a7e61811ef8ae558ce721e2e3fed04ce7a5a5345)

* add content-type: application/json to test HTTP requests

* fix pluginproxy test

* Fix linter when comparing errors

Co-authored-by: Kevin Minehart <kmineh0151@gmail.com>
Co-authored-by: Dan Cech <dcech@grafana.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Serge Zaitsev <serge.zaitsev@grafana.com>
Co-authored-by: Vardan Torosyan <vardants@gmail.com>
2022-02-09 13:44:38 +01:00
85ea1a5d64 Prometheus: Fix Azure authentication support (#44407)
Re-adding back Azure authentication support to Prometheus datasource 
after the datasource query logic was rewritten from plugin.json routes to 
Go backend.

Ref #35857
2022-02-03 17:06:31 +01:00
f6414ea2b2 CodeQL: Try to fix uncontrolled data used in path expression (#43462)
Ref #43080
2022-01-11 17:37:58 +01:00
8927a3ca20 Chore: Query endpoint refactor (#41637)
Get rid of using legacydata.RequestHandler in HTTPServer, /api/tsdb/query and pkg/expr 
with the goal of deprecating /api/tsdb/query and remove it completely eventually. This is 
the first step of cleaning up the HTTP API query endpoint.

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
2021-11-29 14:21:54 +01:00
5652bde447 Encryption: Use secrets service (#40251)
* Use secrets service in pluginproxy

* Use secrets service in pluginxontext

* Use secrets service in pluginsettings

* Use secrets service in provisioning

* Use secrets service in authinfoservice

* Use secrets service in api

* Use secrets service in sqlstore

* Use secrets service in dashboardshapshots

* Use secrets service in tsdb

* Use secrets service in datasources

* Use secrets service in alerting

* Use secrets service in ngalert

* Break cyclic dependancy

* Refactor service

* Break cyclic dependancy

* Add FakeSecretsStore

* Setup Secrets Service in sqlstore

* Fix

* Continue secrets service refactoring

* Fix cyclic dependancy in sqlstore tests

* Fix secrets service references

* Fix linter errors

* Add fake secrets service for tests

* Refactor SetupTestSecretsService

* Update setting up secret service in tests

* Fix missing secrets service in multiorg_alertmanager_test

* Use fake db in tests and sort imports

* Use fake db in datasources tests

* Fix more tests

* Fix linter issues

* Attempt to fix plugin proxy tests

* Pass secrets service to getPluginProxiedRequest in pluginproxy tests

* Fix pluginproxy tests

* Revert using secrets service in alerting and provisioning

* Update decryptFn in alerting migration

* Rename defaultProvider to currentProvider

* Use fake secrets service in alert channels tests

* Refactor secrets service test helper

* Update setting up secrets service in tests

* Revert alerting changes in api

* Add comments

* Remove secrets service from background services

* Convert global encryption functions into vars

* Revert "Convert global encryption functions into vars"

This reverts commit 498eb19859eba364a2400a6d7e73236b1c9a5b37.

* Add feature toggle for envelope encryption

* Rename toggle

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
Co-authored-by: Joan López de la Franca Beltran <joanjan14@gmail.com>
2021-11-04 18:47:21 +02:00
ff086df3b5 Fix decrypting secrets in alerting migration (#41061) 2021-10-29 14:20:07 +03:00
722c414fef Encryption: Refactor securejsondata.SecureJsonData to stop relying on global functions (#38865)
* Encryption: Add support to encrypt/decrypt sjd

* Add datasources.Service as a proxy to datasources db operations

* Encrypt ds.SecureJsonData before calling SQLStore

* Move ds cache code into ds service

* Fix tlsmanager tests

* Fix pluginproxy tests

* Remove some securejsondata.GetEncryptedJsonData usages

* Add pluginsettings.Service as a proxy for plugin settings db operations

* Add AlertNotificationService as a proxy for alert notification db operations

* Remove some securejsondata.GetEncryptedJsonData usages

* Remove more securejsondata.GetEncryptedJsonData usages

* Fix lint errors

* Minor fixes

* Remove encryption global functions usages from ngalert

* Fix lint errors

* Minor fixes

* Minor fixes

* Remove securejsondata.DecryptedValue usage

* Refactor the refactor

* Remove securejsondata.DecryptedValue usage

* Move securejsondata to migrations package

* Move securejsondata to migrations package

* Minor fix

* Fix integration test

* Fix integration tests

* Undo undesired changes

* Fix tests

* Add context.Context into encryption methods

* Fix tests

* Fix tests

* Fix tests

* Trigger CI

* Fix test

* Add names to params of encryption service interface

* Remove bus from CacheServiceImpl

* Add logging

* Add keys to logger

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

* Add missing key to logger

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

* Undo changes in markdown files

* Fix formatting

* Add context to secrets service

* Rename decryptSecureJsonData to decryptSecureJsonDataFn

* Name args in GetDecryptedValueFn

* Add template back to NewAlertmanagerNotifier

* Copy GetDecryptedValueFn to ngalert

* Add logging to pluginsettings

* Fix pluginsettings test

Co-authored-by: Tania B <yalyna.ts@gmail.com>
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2021-10-07 17:33:50 +03:00
62689ec804 Security: Add secrets service (#39418)
* Add secrets service

* Revert accidental changes in util encryption

* Make minor changes

Move functional options to models

Revert renaming types to models

* Add context

* Minor change in GetDataKey

* Use CreateDataKeyWithDBSession in CreateDataKey

* Handle empty DEK name in DeleteDataKey

* Rename defaultProvider

* Remove secrets store service
2021-10-01 15:39:57 +03:00
fc73bc1161 LibraryElements: Enables creating library elements with specific UID (#39019)
* LibraryPanels: Enables create/update library panels with specific UID

* Chore: added check for uid length after PR comments

* Refactor: creates IsShortUIDTooLong function

* Refactor: adds UID to PATCH endpoint

* Refactor: clarifies the patch code

* Refactor: changes  after PR comments
2021-09-10 11:22:13 +02:00
ea8d9d77f4 Admin: User list tweaks (#38750)
* Setup filter

* Enable filtering users by active in last 30 days

* Add loading state

* Update last active age strings

* Tweak user list

* Use theme spacing

* Improve table's accessibility

* Add more aria-labels
2021-09-01 16:53:58 +03:00
8433def04f Encryption: Convert functions into global variables (#37240) 2021-07-30 15:48:27 +02:00
303352a89b Plugins: Ignore symlinked folders when verifying plugin signature (#34434)
* add check + test

* fix test

* add manifest

* fix linter

* add nolint

* separate err cond checks

* only collect relevant plugin files

* skip symlinks

* refactor

* add missing test files + enable scanning Chromium.app/

* remove test since case already covered

* remove unnecessary changes from before

* refactor

* remove comment
2021-06-15 11:55:47 +02:00
5b5cb94809 Fix: fix loading symbolically linked plugins (#35635) 2021-06-15 09:10:30 +03:00
83f26e9ce2 Fix: only recurse a symbolic link if it is a directory (#35455)
* only recurse a symbolic link if it is a directory

* added test for detecting valid plugins using lib dirs with symbolic links in them (like oracle)

* fix linting errors

* added extra checks as per code-review
2021-06-10 10:25:07 -03:00
a5082ab112 Chore: additional check when decrypting values (#34637)
* Chore: additional check when decrypting values

* Apply suggestions from code review
2021-05-25 18:35:54 +03:00