153 Commits

Author SHA1 Message Date
fe18c63204 mysql: do not import httpClient from core grafana (#77628)
mysql: do not use core-only exports
2023-11-13 10:25:19 +01:00
9d92818cae Plugins: Update PDC pattern from latest plugin SDK changes (#76576)
* update with sdk

* do sql

* fix core plugins

* fix proxy settings

* bump SDK version

* tidy

* enable pdc for test

* add codeowners

* bump dep

* go mod tidy

* bump SDK
2023-10-16 16:40:04 +02:00
91cf4f0c1c Revert "Plugins: Update PDC pattern from latest plugin SDK changes" (#76565)
Revert "Plugins: Update PDC pattern from latest plugin SDK changes (#76036)"

This reverts commit 151f6d6216e5e93edf56e4bbc6d3bfa5e0690b45.
2023-10-13 12:35:16 -04:00
151f6d6216 Plugins: Update PDC pattern from latest plugin SDK changes (#76036)
* update with sdk

* do sql

* fix core plugins

* fix proxy settings

* bump SDK version

* tidy

* enable pdc for test

* add codeowners
2023-10-13 14:42:15 +02:00
7fca1bde54 Plugins: Set grafana config, plugin version and user agent on plugin requests (#75171)
* first pass

* fixup

* remove test line

* fix tests

* use new fields

* fix imports + formatting

* fix tests

* rollback changes

* undo whitespace

* apply pr feedback
2023-09-21 11:33:31 +02:00
93cdc94a94 Chore: capitalise logs in other backend packages (#74344)
* capitalise logs in observability logs

* capitalise oss-bit-tent packages

* capitalise logs in aws-datasources

* capitalise logs for traces and profiling

* capitalise logs for partner datasources

* capitalise logs in plugins platform

* capitalise logs for observability metrics
2023-09-04 22:25:43 +02:00
025b2f3011 Chore: use any rather than interface{} (#74066) 2023-08-30 18:46:47 +03:00
4ece133fce Secure socks proxy: use Grafana Plugin SDK (#71616) 2023-07-18 15:23:02 -06:00
66d2214c3b Config: Add configuration option to define custom user-facing general error message for certain error types (#70023)
---------

Co-authored-by: Summer Wollin <summer.wollin@grafana.com>
Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
2023-06-16 10:46:47 -05:00
6758fd4888 MySQL: Add option to allow cleartext passwords (#63232)
* Add "Allow Cleartext Passwords" checkbox to MySQL connection settings

* Fix lint issues

* Add docs

* Add line break and bold text

---------

Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>
2023-05-26 12:33:55 +02:00
286b9e08e9 Plugins: Bump Plugin SDK version and address instance management breaking changes (#68900)
* bump sdk and propagate ctx

* fix tests
2023-05-24 10:19:34 +02:00
b0e681c339 Chore: Update target in go.mod to 1.20 and golangci-lint to v1.52.2 (#67102) 2023-05-10 13:47:35 +02:00
86228ba1a0 Datasource: Fix missing raw SQL query in Query Inspector when query returns zero rows (#67844)
* baldm0mma/issue5799/ add empty frame with all meta data

* baldm0mma/issue5799/ add null test

* write an integration test around query results with empty rows

* baldm0mma/issue5799/ add tests to mssql and postgres

* remove use of apparently reserved keyword in mysql8

* baldm0mma/issue5799

* baldm0mma/issue5799/ update tests

* baldm0mma/issue5799/ update test structs

* baldm0mma/issue5799/ update annotation

---------

Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
2023-05-09 08:29:02 -06:00
255d8f3326 Plugins: Remove secure socks proxy feature toggle (#66611) 2023-04-18 11:11:23 -06:00
92d92187d9 SQL Datasources: Update Max Connection and Max Idle Connection Defaults to 100 and add auto mode (#65834)
* Update connection configuration for SQL datasources

* Working auto state for connection numbers

* Add migration

* Use defaults from constants file

* Remove dead code

* Add tests and restructure useMigrateDatabaseField

* Update function names

* Update docs

* Make sure we don't continually issue updates

* Update docs

* Use onOptionsChnage in ConnectionLimits

* Update docs

* Clean up docs

* Update migration

* Fix default values in docs

* Fix spacing issue

* Fix test

* Update default values for SQL connections

* Include consts

* Allow override for default SQL datasource connection parameters

* Fix linter errors

* Remove extra @ts-ignore

* Centralize logic for default values

* Remove debugging

* Remove unecessary function

* Update configuration docs

* minor suggested change

* Fix comment misspelling

* Remove unecessary default setting code

* Update docs to indicate that code was included for backport version

* Remove dead code

---------

Co-authored-by: lwandz13 <larissa.wandzura@grafana.com>
2023-04-17 11:44:05 +03:00
10db808ea1 Plugins: Add sql support for the secure socks proxy (#64630) 2023-03-16 06:54:19 -06:00
c25fe34ac0 SQL Datasources: Use health check for config test (#59867)
* SQL Datasources: Use health check for config test

* Remove unnecessary test

* Fix test errors

* Revert mysql go driver update

* Use transform query error

* Use TransformQueryError from sql_engine
2022-12-19 17:17:52 +01:00
d65899de7b SQL Datasources: Move database setting to jsonData (#58649)
* Datasource settings: Add deprecation notice for database field

* SQL Datasources: Migrate from settings.database to settings.jsonData.database

* Check jsonData first

* Remove comment from docs
2022-12-13 09:56:52 +01:00
c02003af3c Refactor time durations (#58484)
This change uses `time.Second` in place of `1000 * time.Millisecond` and `time.Minute` in place of `60*time.Second`.
2022-11-22 15:09:15 +08:00
d9c40ca41e Chore: Update sqleng, elasticsearch, tempo and opentsdb plugins to support contextual logs. (#57777)
* make sql engine use pick log context for logs
* update tempo to get log context
* update opentsdb to use log context
* update es client to use log context
2022-11-02 10:03:50 -04:00
05709ce411 chore: remove sqlstore & mockstore dependencies from (most) packages (#57087)
* chore: add alias for InitTestDB and Session

Adds an alias for the sqlstore InitTestDB and Session, and updates tests using these to reduce dependencies on the sqlstore.Store.

* next pass of removing sqlstore imports
* last little bit
* remove mockstore where possible
2022-10-19 09:02:15 -04:00
bd35e6917a Chore: Exclude integration tests from running on test-backend step (#50359)
* Chore: Exclude integration tests from running on test-backend step

* Remove -v from go test command

* Add check to skip integration tests before each integration test

* Try to restart pipeline

* Retrying to make pipeline run
2022-06-10 11:46:21 -04:00
c63ebc887b Chore: Run integration tests without grabpl (#49448)
* Chore: Run integration tests without grabpl

* Add new step for integration tests in lib.star

* Remove old integration test step from lib.star

* Change drone signature

* Fix: Edit starlark integration step to not affect enterprise

* Remove all build tags & rename starlark integration test step

* Resync .drone.yml with .drone.star

* Fix lint errors

* Fix lint errors

* Fix lint errors

* Fix more lint errors

* Fix another lint error

* Rename integration test step

* Fix last lint error

* Recomment enterprise step

* Remove comment from Makefile

Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
2022-06-01 14:55:22 -04:00
50c2b4682a Chore: Rename integration tests (#49438)
* Chore: Rename integration tests

* Remove one Integration

Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
2022-05-24 11:04:03 +02:00
76da017d00 MySQL: Prevent pre-epoch timestamps from breaking __timeFilter (#45345)
* add logic to change timeFilter query when FROM time is pre-EPOCH

* fix timeFilter tests where FROM time is pre-EPOCH
2022-02-22 20:06:04 +01:00
7fbc7d019a Plugins: Core plugins register via backend factory provider (#43171)
* refactoring store interface and init flow

* fix import

* fix linter

* refactor resource calling

* load with class

* re-order args

* fix tests

* fix linter

* remove old creator

* add custom config struct

* fix some tests

* cleanup

* fix

* tackle plugins

* fix linter

* refactor and fix test

* add connect failure error

* add fix for azure, cloud monitoring and test data

* restructure

* remove unused err

* add fake tracer for test

* fix grafana ds plugin
2022-01-20 18:16:22 +01:00
e4ba5f17dd add core plugins thru store (#43085) 2021-12-14 15:22:40 +01:00
b80fbe03f0 Plugins: Refactor Plugin Management (#40477)
* add core plugin flow

* add instrumentation

* move func

* remove cruft

* support external backend plugins

* refactor + clean up

* remove comments

* refactor loader

* simplify core plugin path arg

* cleanup loggers

* move signature validator to plugins package

* fix sig packaging

* cleanup plugin model

* remove unnecessary plugin field

* add start+stop for pm

* fix failures

* add decommissioned state

* export fields just to get things flowing

* fix comments

* set static routes

* make image loading idempotent

* merge with backend plugin manager

* re-use funcs

* reorder imports + remove unnecessary interface

* add some TODOs + remove unused func

* remove unused instrumentation func

* simplify client usage

* remove import alias

* re-use backendplugin.Plugin interface

* re order funcs

* improve var name

* fix log statements

* refactor data model

* add logic for dupe check during loading

* cleanup state setting

* refactor loader

* cleanup manager interface

* add rendering flow

* refactor loading + init

* add renderer support

* fix renderer plugin

* reformat imports

* track errors

* fix plugin signature inheritance

* name param in interface

* update func comment

* fix func arg name

* introduce class concept

* remove func

* fix external plugin check

* apply changes from pm-experiment

* fix core plugins

* fix imports

* rename interface

* comment API interface

* add support for testdata plugin

* enable alerting + use correct core plugin contracts

* slim manager API

* fix param name

* fix filter

* support static routes

* fix rendering

* tidy rendering

* get tests compiling

* fix install+uninstall

* start finder test

* add finder test coverage

* start loader tests

* add test for core plugins

* load core + bundled test

* add test for nested plugin loading

* add test files

* clean interface + fix registering some core plugins

* refactoring

* reformat and create sub packages

* simplify core plugin init

* fix ctx cancel scenario

* migrate initializer

* remove Init() funcs

* add test starter

* new logger

* flesh out initializer tests

* refactoring

* remove unused svc

* refactor rendering flow

* fixup loader tests

* add enabled helper func

* fix logger name

* fix data fetchers

* fix case where plugin dir doesn't exist

* improve coverage + move dupe checking to loader

* remove noisy debug logs

* register core plugins automagically

* add support for renderer in catalog

* make private func + fix req validation

* use interface

* re-add check for renderer in catalog

* tidy up from moving to auto reg core plugins

* core plugin registrar

* guards

* copy over core plugins for test infra

* all tests green

* renames

* propagate new interfaces

* kill old manager

* get compiling

* tidy up

* update naming

* refactor manager test + cleanup

* add more cases to finder test

* migrate validator to field

* more coverage

* refactor dupe checking

* add test for plugin class

* add coverage for initializer

* split out rendering

* move

* fixup tests

* fix uss test

* fix frontend settings

* fix grafanads test

* add check when checking sig errors

* fix enabled map

* fixup

* allow manual setup of CM

* rename to cloud-monitoring

* remove TODO

* add installer interface for testing

* loader interface returns

* tests passing

* refactor + add more coverage

* support 'stackdriver'

* fix frontend settings loading

* improve naming based on package name

* small tidy

* refactor test

* fix renderer start

* make cloud-monitoring plugin ID clearer

* add plugin update test

* add integration tests

* don't break all if sig can't be calculated

* add root URL check test

* add more signature verification tests

* update DTO name

* update enabled plugins comment

* update comments

* fix linter

* revert fe naming change

* fix errors endpoint

* reset error code field name

* re-order test to help verify

* assert -> require

* pm check

* add missing entry + re-order

* re-check

* dump icon log

* verify manager contents first

* reformat

* apply PR feedback

* apply style changes

* fix one vs all loading err

* improve log output

* only start when no signature error

* move log

* rework plugin update check

* fix test

* fix multi loading from cfg.PluginSettings

* improve log output #2

* add error abstraction to capture errors without registering a plugin

* add debug log

* add unsigned warning

* e2e test attempt

* fix logger

* set home path

* prevent panic

* alternate

* ugh.. fix home path

* return renderer even if not started

* make renderer plugin managed

* add fallback renderer icon, update renderer badge + prevent changes when renderer is installed

* fix icon loading

* rollback renderer changes

* use correct field

* remove unneccessary block

* remove newline

* remove unused func

* fix bundled plugins base + module fields

* remove unused field since refactor

* add authorizer abstraction

* loader only returns plugins expected to run

* fix multi log output
2021-11-01 10:53:33 +01:00
249856e527 Chore: Refactor GoConvey in mysql package tests (#40855) 2021-10-26 13:18:17 +02:00
722c414fef Encryption: Refactor securejsondata.SecureJsonData to stop relying on global functions (#38865)
* Encryption: Add support to encrypt/decrypt sjd

* Add datasources.Service as a proxy to datasources db operations

* Encrypt ds.SecureJsonData before calling SQLStore

* Move ds cache code into ds service

* Fix tlsmanager tests

* Fix pluginproxy tests

* Remove some securejsondata.GetEncryptedJsonData usages

* Add pluginsettings.Service as a proxy for plugin settings db operations

* Add AlertNotificationService as a proxy for alert notification db operations

* Remove some securejsondata.GetEncryptedJsonData usages

* Remove more securejsondata.GetEncryptedJsonData usages

* Fix lint errors

* Minor fixes

* Remove encryption global functions usages from ngalert

* Fix lint errors

* Minor fixes

* Minor fixes

* Remove securejsondata.DecryptedValue usage

* Refactor the refactor

* Remove securejsondata.DecryptedValue usage

* Move securejsondata to migrations package

* Move securejsondata to migrations package

* Minor fix

* Fix integration test

* Fix integration tests

* Undo undesired changes

* Fix tests

* Add context.Context into encryption methods

* Fix tests

* Fix tests

* Fix tests

* Trigger CI

* Fix test

* Add names to params of encryption service interface

* Remove bus from CacheServiceImpl

* Add logging

* Add keys to logger

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

* Add missing key to logger

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

* Undo changes in markdown files

* Fix formatting

* Add context to secrets service

* Rename decryptSecureJsonData to decryptSecureJsonDataFn

* Name args in GetDecryptedValueFn

* Add template back to NewAlertmanagerNotifier

* Copy GetDecryptedValueFn to ngalert

* Add logging to pluginsettings

* Fix pluginsettings test

Co-authored-by: Tania B <yalyna.ts@gmail.com>
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2021-10-07 17:33:50 +03:00
64c8d32fe7 Use sdk pkg for gtime (#39354) 2021-09-21 13:08:52 +02:00
1c892a2fc4 Postgres/MySQL/MSSQL: Add setting to limit maximum amount of rows processed (#38986)
Adds a new setting dataproxy.row_limit that allows an operator to limit the 
amount of rows being processed/accepted in response to database queries 
originating from SQL data sources.

Closes #38975
Ref #39095

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
2021-09-13 15:27:51 +02:00
fbdaf56a84 Postgres/MySQL/MSSQL: Fix region annotations not displayed correctly (#38936)
Fix region annotations not displayed correctly when returning timeend column 
as epoch timestamp and by that making sure that the returned data frame field 
named timeend is treated as time type.

Fixes #38533
2021-09-08 01:54:48 +02:00
c989f0fbbe SQL: Migrate to use SDK contracts (#36635)
* convert SQLs to use sdk contracts

* make draft

* postgres

* intermedia

* get datasourceinfo filled at the beginning of the service

* move the interval into package because of cyclict  import and fix all postgres tests

* fix mysql test

* fix mssql

* fix the test for pr https://github.com/grafana/grafana/issues/35839

* fix some issue about intervalv2 package

* update sql test

* wire migration for SQLs

* add sqls to the background process

* make it register instead of register and start

* revert formatting

* fix tests

* fix linter

* remove integration test

* Postgres test fix

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-09-07 09:35:37 +02:00
643c7fa0cb Chore: update all +build statements (#38782) 2021-09-01 17:38:56 +03:00
10c892fa5b Postgres/MySQL/MSSQL: Fix name of time field should be named Time for time series queries (#36720)
Name of time field changed in v8 for time series queries from Time to the name of the selected 
time column, i.e. time or time_sec. These changes should make sure that the name of time field 
is always returned as Time for time series queries.

Fixes #36059

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2021-07-14 11:29:51 +02:00
73fdd9dc22 [MYSQL] convert explicitely smallint and tinyint (#35897)
* [MYSQL] convert explicitely smallint and tinyint

* clean up code

* fix comments
2021-06-22 00:10:45 +08:00
1d6e99bc95 SQL: Fixes issues with showing value column name prefix in legends (#35839)
* rename strategy

* Update pkg/tsdb/sqleng/sql_engine.go

Co-authored-by: Torkel Ödegaard <torkel@grafana.org>

* more strict constraints

* Fixed so that it works on multi series results

* only apply the logic when original query returns 3 fields

* removed part of comment

* Update mysql test

Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Torkel Ödegaard <torkel@grafana.com>
2021-06-18 08:05:23 +02:00
0611207f3b Postgres/MySQL/MSSQL: Fix concurrent map writes panic (#35510)
Fixes panic/fatal error concurrent map writes in SQL data sources when multiple 
queries are executed concurrently and you interpolate SQL query before executing it.

Fixes #35469
2021-06-11 14:56:29 +02:00
29be741f26 MySQL Datasource: Add timezone parameter (#27535)
* Add MySQL timezone parameter

* Reword docs/sources/features/datasources/mysql.md

Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>

* update documentation

* fix formatting mistake from merge

* enhance documentation with reason for parameter usage

* enhance documentation with reason for parameter usage

Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
2021-05-27 10:03:03 +02:00
348e76fc8e Datasource: Shared HTTP client provider for core backend data sources and any data source using the data source proxy (#33439)
Uses new httpclient package from grafana-plugin-sdk-go introduced 
via grafana/grafana-plugin-sdk-go#328. 
Replaces the GetHTTPClient, GetTransport, GetTLSConfig methods defined 
on DataSource model.
Longer-term the goal is to migrate core HTTP backend data sources to use the 
SDK contracts and using httpclient.Provider for creating HTTP clients and such.

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-05-19 23:53:41 +02:00
6c728d6a78 SQL: Use SDK sqlutil package refactor (#33758)
Upgrades grafana-plugin-sdk-go dependency to v0.94.0 to get 
the refactored sqlutil package. 

Co-authored-by: Kevin Minehart <kmineh0151@gmail.com>
2021-05-11 14:59:33 +02:00
bd66c8dde3 SQL data sources: Convert to return data frames (#32257)
Convert SQL data sources to return data frames.

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Hugo Häggmark <hugo.haggmark@gmail.com>
2021-05-05 16:46:07 +02:00
7627b55ef4 TSDB: add deprecation comments to many tsdb structs (#33281) 2021-04-23 03:03:11 +02:00
b79e61656a Introduce TSDB service (#31520)
* Introduce TSDB service

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>
2021-03-08 07:02:49 +01:00
9593d57914 Chore: Enable errorlint linter (#29227)
* Enable errorlint linter
* Handle wrapped errors

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2020-11-19 14:47:17 +01:00
2bf964c61e Chore: Fix linting issues caught by ruleguard (#28799)
* Chore: Fix linting issues caught by ruleguard

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Improve error check

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-11-17 11:27:45 +01:00
b5379c5335 Chore: Fix SQL related Go variable naming (#28887)
* Chore: Fix variable naming

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-11-11 06:21:08 +01:00
574553ec7b Chore: Fix issues found by staticcheck (#28802)
* Fix linting issues

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-11-05 11:29:39 +01:00
a078e40238 Settings: Rename constants/variables to follow Go naming standards (#28002)
* settings: Rename constants/variables to follow Go naming standards
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-10-02 15:45:45 +02:00