55 Commits

Author SHA1 Message Date
2afcb0e95b Data Sources: Fix namespace (#107124)
Fix namespace
2025-06-24 15:40:27 +01:00
8f79e4882f Replace usage of http.DefaultClient and http.DefaultTransport (#104135)
Remove usage of http.DefaultClient and http.DefaultTransport

Part of grafana/data-sources#484
2025-05-09 13:26:39 -04:00
35145801a2 Data Sources: Add gauge for response size (#104394) 2025-04-24 13:26:28 +01:00
757be6365a CI: Bump golangci-lint to 2.0.2 (#103572) 2025-04-10 14:42:23 +02:00
9eaaf95701 Hackaton 12: Add some unit tests, take 1 (#101457)
* tsdb/graphite: add happy path test for QueryData

* infra/httpclient/httpclientprovider: add tests for RedirectLimitMiddleware
2025-02-28 16:11:11 +02:00
d192a44469 Plugins: Modify interface for plugin validations to allow taking PDC into account (#96089)
* Request interceptor: Do not block PDC

* Apply change after feedback received

* Add test

* Check if secure socks proxy configured for the instance

* Apply suggestions from code review

* Add dedicated service for datasource request URL validation (#99179)

---------

Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2025-01-24 17:01:46 +02:00
b28085110d Plugins: Auto instrumentation improvements (#94193) 2024-10-15 14:02:34 +02:00
abca0380a8 use count_bytes_reader from plugin-sdk-go (#92553)
* use count_bytes_reader from plugin-sdk-go

* run `make update-workspace`

* update postgres tests

* update mysql tests

* time back to utc

* make update-workspace done

---------

Co-authored-by: Kyle Brandt <kyle@grafana.com>
2024-08-28 16:44:13 +02:00
3ac109e759 AWS: Update deprecated aws-sdk functions from env variable versions (#89643) 2024-06-26 11:56:39 -04:00
961272a76d Datasources: Enable native histograms for datasource response size metric (#87962) 2024-05-16 13:48:51 +02:00
7b392d40a0 Auth: Sign sigV4 request after adding headers (#87063) 2024-04-29 13:07:45 -04:00
6204f1e847 Chore: Use SigV4 middleware from aws-sdk (#84462) 2024-03-18 09:33:22 +01:00
c061cc33cc Chore: Use response limit middleware from SDK (#83915) 2024-03-13 10:14:16 +01:00
2c5b72e844 AuthZ: add headers for IP range AC checks for data source proxy requests (#81662)
* add a middleware that appens headers for IP range AC to data source proxy requests

* update code

* add tests

* fix a mistake

* add logging

* refactor to reuse code

* small cleanup

* skip the plugins middleware if the header is already set

* skip the plugins middleware if the header is already set
2024-03-06 12:40:48 +00:00
89fb56bc11 Add secureSocksDSProxyEnabled label to grafana_datasource_request_duration_seconds (#80910)
* Add secureSocksDSProxyEnabled label to grafana_datasource_request_duration_seconds

* rename label from secureSocksDSProxyEnabled to secure_socks_ds_proxy_enabled

* DataSourceMetricsMiddleware: add secure_socks_ds_proxy_enabled label to every metric
2024-02-01 11:42:43 -03:00
ef60c90dfa Plugins: Make file system path handling in tests OS agnostic (#79651)
* make path handling OS agnostic

* PR feedback

* fix input for test case
2023-12-19 11:01:48 +01:00
e4c1a7a141 Tracing: Standardize on otel tracing (#75528) 2023-10-03 14:54:20 +02:00
5cd1541aee HTTP Client Metrics: Add datasource_type to datasource metrics (#72130)
Metrics: Add datasource_type to datasource metrics
2023-07-24 14:22:08 +02:00
4ece133fce Secure socks proxy: use Grafana Plugin SDK (#71616) 2023-07-18 15:23:02 -06:00
680fc85cdf Chore: Delete unused cookie forward middleware (#70403)
Delete unused cookie forward middleware
2023-06-21 22:39:10 +03:00
255d8f3326 Plugins: Remove secure socks proxy feature toggle (#66611) 2023-04-18 11:11:23 -06:00
a89202eab2 Plugins: Improve instrumentation by adding metrics and tracing (#61035)
* WIP: Plugins tracing

* Trace ID middleware

* Add prometheus metrics and tracing to plugins updater

* Add TODOs

* Add instrumented http client

* Add tracing to grafana update checker

* Goimports

* Moved plugins tracing to middleware

* goimports, fix tests

* Removed X-Trace-Id header

* Fix comment in NewTracingHeaderMiddleware

* Add metrics to instrumented http client

* Add instrumented http client options

* Removed unused function

* Switch to contextual logger

* Refactoring, fix tests

* Moved InstrumentedHTTPClient and PrometheusMetrics to their own package

* Tracing middleware: handle errors

* Report span status codes when recording errors

* Add tests for tracing middleware

* Moved fakeSpan and fakeTracer to pkg/infra/tracing

* Add TestHTTPClientTracing

* Lint

* Changes after PR review

* Tests: Made "ended" in FakeSpan private, allow calling End only once

* Testing: panic in FakeSpan if span already ended

* Refactoring: Simplify Grafana updater checks

* Refactoring: Simplify plugins updater error checks and logs

* Fix wrong call to checkForUpdates -> instrumentedCheckForUpdates

* Tests: Fix wrong call to checkForUpdates -> instrumentedCheckForUpdates

* Log update checks duration, use Info log level for check succeeded logs

* Add plugin context span attributes in tracing_middleware

* Refactor prometheus metrics as httpclient middleware

* Fix call to ProvidePluginsService in plugins_test.go

* Propagate context to update checker outgoing http requests

* Plugin client tracing middleware: Removed operation name in status

* Fix tests

* Goimports tracing_middleware.go

* Goimports

* Fix imports

* Changed span name to plugins client middleware

* Add span name assertion in TestTracingMiddleware

* Removed Prometheus metrics middleware from grafana and plugins updatechecker

* Add span attributes for ds name, type, uid, panel and dashboard ids

* Fix http header reading in tracing middlewares

* Use contexthandler.FromContext, add X-Query-Group-Id

* Add test for RunStream

* Fix imports

* Changes from PR review

* TestTracingMiddleware: Changed assert to require for didPanic assertion

* Lint

* Fix imports
2023-03-28 11:01:06 +02:00
c0b5d2dfaa Tracing: Support multiple OTel propagators (#61199)
* tracing: Support multiple OTel propagators

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* tracing: add TraceIDString method to fix up tests

This method will be useful elsewhere if we want to log the trace ID.

* improve propagation docs

Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>

* doc style fix

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* Use tracing.TraceIDFromContext instead of adding TraceIDString method

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

---------

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
Co-authored-by: Bryan Boreham <bjboreham@gmail.com>
Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
2023-03-27 14:56:24 -04:00
7731a4d6af Tracing: Add more detail to HTTP Outgoing Request (#64757)
Add more detail to HTTP Outgoing Request.
As events rather than sub-spans, to save cluttering the view.
2023-03-27 13:04:43 +02:00
10db808ea1 Plugins: Add sql support for the secure socks proxy (#64630) 2023-03-16 06:54:19 -06:00
3c218d742f Datasources: Add user_agent header customization for outgoing HTTP requests (#63769) 2023-02-28 14:10:05 +01:00
fe27acc3a9 chore: move validations model into the validations service (#61953) 2023-01-23 15:10:14 -05:00
c35c689a96 Plugins: Automatically forward plugin request HTTP headers in outgoing HTTP requests (#60417)
Automatically forward core plugin request HTTP headers in outgoing HTTP requests. 
Core datasource plugin authors don't have to specifically handle forwarding of HTTP 
headers, e.g. do not have to "hardcode" the header-names in the datasource plugin, 
if not having custom needs.

Fixes #57065
2022-12-21 13:25:58 +01:00
6dbe3b555f Plugins: Refactor forward of cookies, OAuth token and header modifications by introducing client middlewares (#58132)
Adding support for backend plugin client middlewares. This allows headers in outgoing 
backend plugin and HTTP requests to be modified using client middlewares.

The following client middlewares added:
Forward cookies: Will forward incoming HTTP request Cookies to outgoing plugins.Client 
and HTTP requests if the datasource has enabled forwarding of cookies (keepCookies).
Forward OAuth token: Will set OAuth token headers on outgoing plugins.Client and HTTP 
requests if the datasource has enabled Forward OAuth Identity (oauthPassThru).
Clear auth headers: Will clear any outgoing HTTP headers that was part of the incoming 
HTTP request and used when authenticating to Grafana.
The current suggested way to register client middlewares is to have a separate package, 
pluginsintegration, responsible for bootstrap/instantiate the backend plugin client with 
middlewares and/or longer term bootstrap/instantiate plugin management. 

Fixes #54135
Related to #47734
Related to #57870
Related to #41623
Related to #57065
2022-12-01 19:08:36 +01:00
6805c951e9 Plugins: add option to proxy ds connections through a secure socks proxy (#59254)
* Plugins: add feature to proxy data source connections
2022-11-29 23:50:59 -06:00
6f8fcae01b [main] Plugin fixes (#57399)
* Plugins: Remove support for V1 manifests

* Plugins: Make proxy endpoints not leak sensitive HTTP headers

* Security: Fix do not forward login cookie in outgoing requests

(cherry picked from commit 4539c33fce5ef23badb08ebcbc09cb0cecb1f539)

Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-10-21 13:54:55 +02:00
Jo
062d255124 Handle ioutil deprecations (#53526)
* replace ioutil.ReadFile -> os.ReadFile

* replace ioutil.ReadAll -> io.ReadAll

* replace ioutil.TempFile -> os.CreateTemp

* replace ioutil.NopCloser -> io.NopCloser

* replace ioutil.WriteFile -> os.WriteFile

* replace ioutil.TempDir -> os.MkdirTemp

* replace ioutil.Discard -> io.Discard
2022-08-10 15:37:51 +02:00
b5d57c45e3 Plugins: Add support for HTTP logger (#46578) 2022-07-21 09:46:47 -04:00
096330c53b datasource metrics: Correct help text for histograms (#50942)
In #50420 a few summary metrics were converted to histograms,
but the Help text in a couple of them still referred to summaries.
This fixes that help text.

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
2022-06-16 14:18:09 +02:00
68691d7775 Convert some metrics to Histograms (#50420)
Because Summary metrics can not be aggreated, convert them to histograms
so that users with HA deployments can use these metrics.
* Convert metrics registration to promauto.
* Improve help text style.

Signed-off-by: SuperQ <superq@gmail.com>
2022-06-15 13:19:43 +02:00
ae9491c3a7 Chore: Make test tracer noop and return no errors (#50797) 2022-06-15 12:40:41 +02:00
99db588919 Data Sources: Add QueryData OAuth & cookie forwarding middleware (#50466) 2022-06-10 08:41:29 -04:00
cddab15769 Chore: Upgrade grafana-plugin-sdk-go to v0.137.0 (#50443)
Co-authored-by: Todd Treece <todd.treece@grafana.com>
2022-06-09 10:10:58 +02:00
aad2983350 Security: fixes CVE-2022-29170 (#49240)
* Request interceptor: block redirects

* handle location missing

* Update pkg/infra/httpclient/httpclientprovider/host_redirect_validation_middleware.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Update pkg/infra/httpclient/httpclientprovider/host_redirect_validation_middleware.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* linter

* fixes tests

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-05-19 17:01:09 +02:00
656ade9884 Shared Azure middleware between Azure Monitor and Prometheus datasources (#46002)
* Scopes in Azure middleware

* Enable Azure middleware without feature flag

* Use common Azure middleware in Azure Monitor

* Apply feature flag to JsonData configuration of Azure auth

* Enforce feature flag in Prometheus datasource

* Prometheus provider tests

* Datasource service tests

* Fix http client provider tests

* Pass sdkhttpclient.Options by reference

* Add middleware to httpclient.Options

* Remove dependency on Grafana settings

* Unit-tests updated

* Fix ds_proxy_test

* Fix service_test
2022-04-01 13:26:49 +02:00
fc42dfe396 Auth: Guarantee consistency of signed SigV4 headers (#45054)
* use latest sigv4 changes

* update configuration docs

* lint

* reformat lint ignore

* specific version for docs
2022-02-08 14:48:17 +01:00
5d66194ec5 FeatureFlags: define features outside settings.Cfg (take 3) (#44443) 2022-01-26 09:44:20 -08:00
cb09162cde Datasources: Simplify the AzureCredentials structure in datasource config (#39209)
Related #35857

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-01-25 14:23:32 +01:00
65bdb3a899 FeatureFlags: Revert managing feature flags outside of settings.Cfg (#44382)
* Revert "FeatureToggles: register all enterprise feature toggles (#44336)"

This reverts commit f53b3fb0071c0d6d16a80d5e172a425aa3f72ca9.

* Revert "FeatureFlags: manage feature flags outside of settings.Cfg (#43692)"

This reverts commit f94c0decbd302140fffe351db200634a5c728545.
2022-01-24 16:08:05 +01:00
f94c0decbd FeatureFlags: manage feature flags outside of settings.Cfg (#43692) 2022-01-20 13:42:05 -08:00
30aa24a183 Chore: Implement OpenTelemtry in Grafana (#42674)
* Separate Tracer interface to TracerService and Tracer

* Fix lint

* Fix:Make it possible to start spans for both opentracing and opentelemetry in ds proxy

* Add span methods, use span interface for rest of tracing

* Fix logs in tracing

* Fix tests that are related to tracing

* Fix resourcepermissions test

* Fix some tests

* Fix more tests

* Add TracingService to wire cli runner

* Remove GlobalTracer from bus

* Renaming test function

* Remove GlobalTracer from TSDB

* Replace GlobalTracer in api

* Adjust tests to the InitializeForTests func

* Remove GlobalTracer from services

* Remove GlobalTracer

* Remove bus.NewTest

* Remove Tracer interface

* Add InitializeForBus

* Simplify tests

* Clean up tests

* Rename TracerService to Tracer

* Update pkg/middleware/request_tracing.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Initialize tracer before passing it to SQLStore initialization in commands

* Remove tests for opentracing

* Set span attributes correctly, remove unnecessary trace initiliazation form test

* Add tracer instance to newSQLStore

* Fix changes due to rebase

* Add modified tracing middleware test

* Fix opentracing implementation tags

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-01-20 11:10:12 +01:00
3be452f995 DataProxy: Fix issue overriding response body when response status is 101 (#41364)
When a request going through Grafana data source proxy responds with a websocket 
upgrade response we cannot override the response body since it produces an error. 
This problem seems to have been introduced in Grafana v8.0 by #38962. 
In addition #40303 added same problem.

Fixes #41292
2021-11-09 14:33:54 +01:00
889d4683a1 Datasources: Set response size metric based on actual bytes read (#40303)
Fixes #33372
2021-10-25 11:49:49 +02:00
ba9d5540b8 Datasources: Introduce response_limit for datasource responses (#38962)
* Introduce response_limit for datasource responses

* Fix lint

* Fix tests

* Add case where limit <= 0 - added parametrized tests

* Add max_bytes_reader.go

* Use new httpclient.MaxBytesReader instead of net/http one

* Fixes according to reviewer's comments

* Add tests for max_bytes_reader

* Add small piece in configuration.md

* Further fixes according to reviewer's comments

* Fix linting - fix test
2021-09-10 16:51:06 +03:00
78596a6756 Migrate to Wire for dependency injection (#32289)
Fixes #30144

Co-authored-by: dsotirakis <sotirakis.dim@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Leon Sorokin <leeoniya@gmail.com>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: spinillos <selenepinillos@gmail.com>
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
2021-08-25 15:11:22 +02:00