35 Commits

Author SHA1 Message Date
069fb0cf38 HTTP Client: Introduce dataproxy_max_idle_connections config variable (#35864)
* Introduce dataproxy_max_idle_connections config var

* Fix according to reviewer's comments

* Fix according to reviewer's comments - round 2

* Remove unused const

* Bring back MaxIdleConnsPerHost

* Fixes according to reviewer's comments
2021-07-07 13:13:53 +03:00
6531424c72 Datasource: Add support for max_conns_per_host in dataproxy settings (#35520)
Allow configuring dataproxy.max_conns_per_host for HTTP data sources.

Ref #35519
Ref #35257
2021-06-11 14:18:08 +02:00
d15d87db8a Datasource: Fix storing timeout option as numeric (#35441)
#31871 introduced support for configuring timeout in seconds
for HTTP data sources. That had a bug where backend expected
a numeric timeout value where it was actually stored as a
string. This should resolve this by requiring input to be
numbers, storing input as numeric and falling back to string
value if there's no numeric value.

Ref #31871
2021-06-10 10:27:14 +02:00
f76f426da3 Chore: Refactor Prometheus HTTP client middleware (#34473)
Following #33439 this refactors the Prometheus HTTP transport 
which is replaced by HTTP client middleware.
2021-05-27 12:43:21 +02:00
91657dad18 HTTP Client: Make ResponseHeaderTimeout default timeout in http client (#34597)
* HTTP Client: Add `ResponseHeaderTimeout` - split from `DialContext` timeout

* Fixes according to reviewer's comments

* Use grafana-plugin-sdk-go v0.100.0
2021-05-25 11:32:41 +03:00
348e76fc8e Datasource: Shared HTTP client provider for core backend data sources and any data source using the data source proxy (#33439)
Uses new httpclient package from grafana-plugin-sdk-go introduced 
via grafana/grafana-plugin-sdk-go#328. 
Replaces the GetHTTPClient, GetTransport, GetTLSConfig methods defined 
on DataSource model.
Longer-term the goal is to migrate core HTTP backend data sources to use the 
SDK contracts and using httpclient.Provider for creating HTTP clients and such.

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-05-19 23:53:41 +02:00
8e70d5d181 Add timeout option to datasource config (#31871) 2021-04-07 10:46:19 -04:00
3b7b49a4aa Auth: Use SigV4 lib from grafana-aws-sdk (#30713)
* replace with lib

* remove test + apply feedback
2021-02-01 16:07:27 +01:00
f021f97c31 DataSourceSettings: Add servername field to DataSource TLS config (#29279)
* DataSourceSettings: Add servername field to DataSource TLS config

A DNS lookup URL can be provided in the DataSource URL field in order
to dynamically load balance between multiple instances of a DataSource.
When using mutual TLS, Golang's TLS config implementation checks that
the certificate's common name (< 1.15) or subject alternative name
(>= 1.15) has the same value as the domain being accessed. If the DNS
entry is dynamically generated for a specific environment, the
certificate cannot be generated with a name matching the dynamic
DNS URL. As such, Golang offers a servername field that can be set to
overwrite what value is used when checking against the certificate's
common name (or subject alternative name). Without this change, Skip
TLS Verify must be set to true in order for the DataSource to work,
removing some of the benefits gained by using mutual TLS.

This commit adds the ability to set Grafana's internal Golang TLS config
servername field from the UI or a provisioned DataSource.

The servername field is optional and the existing behavior is retained if
the field is not set.

Co-authored-by: Dana Pruitt <dpruitt@vmware.com>
Co-authored-by: Jeremy Alvis <jalvis@pivotal.io>

* Update docs with PR review changes

Co-authored-by: Jeremy Alvis <jalvis@pivotal.io>
Co-authored-by: Dana Pruitt <dpruitt@vmware.com>

* Update with additional PR requested changes

* Minor updates based on PR change requests

Co-authored-by: Dana Pruitt <dpruitt@vmware.com>
2020-12-10 16:07:05 +01:00
3d3a7cbba8 Chore: Fix staticcheck issues (#28860)
* Chore: Fix issues reported by staticcheck

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Undo changes

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Chore: Fix issues reported by staticcheck

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix test

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix test

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-11-05 13:07:06 +01:00
4d2b20f727 Auth: Fix SigV4 request verification step for Amazon Elasticsearch Service (#28481)
* remove forward header and add extra steps for encoding

* add comment

* re-use forwarded header

* fix service param

* Make SDK the default auth option in UI

* use SDK code instead

* propagate err

* refactor flow

* fix prom service namespace
2020-10-26 12:36:11 +01:00
519ec93c7d Datasource: Prefix SigV4 json data keys with sigV4 (#28195)
Prefixes SigV4 json data and secure json data keys with sigV4 to make 
sure they don't collide with any other settings. Also minor fixes to UI.

Ref #27552
2020-10-12 17:52:17 +02:00
92c2a6c239 Fix: Add additional settings for dataproxy to help with network proxy timeouts (#27841)
* adding additional settings for datasource cache transport

* added documentation for the new changes

* fixing small typo in defaults.ini comment

* fixing small typo in configuration.md comment

* Update conf/defaults.ini keepalive comment per review

Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>

* Update conf/defaults.ini idle conn comment per review

Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>

* Update conf/defaults.ini anon user comment per review

Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>

* Update docs/sources/administration/configuration.md idle conn comment per review

Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>

* adding suggestions from papagian

* fixing configuration.md

* fixing configuration.md typo

* Apply suggestions from code review aknuds1

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* updating sample.ini

* Apply suggestions for docs from code review papagian

Co-authored-by: Sofia Papagiannaki <papagian@users.noreply.github.com>

* Update docs/sources/administration/configuration.md fix typo

Co-authored-by: Sofia Papagiannaki <papagian@users.noreply.github.com>

Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Sofia Papagiannaki <papagian@users.noreply.github.com>
2020-10-12 11:36:47 +03:00
7d63b2c473 Auth: Add Sigv4 auth option to datasources (#27552)
* create transport chain

* add frontend

* remove log

* inline field updates

* allow ARN, Credentials + Keys auth in frontend

* configure credentials

* add tests and refactor

* update frontend json field names

* fix tests

* fix comment

* add app config flag

* refactor tests

* add return field for tests

* add flag for UI display

* update comment

* move logic

* fix config

* pass config through props

* update docs

* pr feedback and add docs coverage

* shorten settings filename

* fix imports

* revert docs changes

* remove log line

* wrap up next as round tripper

* only propagate required config

* remove unused import

* remove ARN option and replace with default chain

* make ARN role assume as supplemental

* update docs

* refactor flow

* sign body when necessary

* remove unnecessary wrapper

* remove newline

* Apply suggestions from code review

* PR fixes

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-10-08 10:03:20 +02:00
747513d444 Plugins: Fix loading of backend plugins (#27951)
Ref #27921
2020-10-01 10:39:42 +02:00
7b891d10ee Instrumentation: Removes invalid chars from label names (#27921) 2020-09-30 20:12:57 +02:00
f529223455 Fix instrumentation panic if there is no response (#27567) 2020-09-14 14:05:47 +03:00
f9752f0e7a Instrumentation: Adds instrumentation for outgoing datasource requests (#27427) 2020-09-09 07:47:05 +02:00
c63bbbaec3 InfluxDB: Close Flux query results (#26917)
* InfluxDB: Drop ctxhttp usage
* InfluxDB: Clean up code
* InfluxDB: Close query results

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-08-13 17:50:53 +02:00
460b01f1fe Datasource: Make sure data proxy timeout applies to HTTP client (#25865)
For backend data sources executing in the backend (not through data proxy) make 
sure that the timeout applies to cached HTTP client.

Fixes #25863

Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
2020-06-29 11:22:49 +02:00
3fc5f4552a CloudWatch: Fix high CPU load (#20579)
* Cache decrypted securejsondata
* Models: Add datasource cache tests
2019-11-22 14:21:23 +01:00
8cd93f0b88 Datasource: Add custom headers on alerting queries (#19508)
* Add custom headers on alerting queries

Reference issue #15381

Signed-off-by: Martin Schneppenheim <martin.schneppenheim@rewe-digital.com>

* Fix datasource transport tests

* Migrate decrypting header test to models pkg

* Check correct header

* Add HTTP transport test

Fixes #15381
2019-10-11 14:28:52 +02:00
a86b8c8a04 http: remove dualstack since its deprecated (#16940)
Deprecated: Fast Fallback is enabled by default. To
disable, set FallbackDelay to a negative value.

https://golang.org/src/net/dial.go
2019-05-08 10:37:48 +02:00
a1cd550df4 revert ds_proxy timeout and implement dataproxy timeout correctly 2019-02-11 13:42:05 +01:00
f157c19e16 extract parsing of datasource tls config to method 2019-01-28 19:38:56 +01:00
ce941a004d fix unconvert issues 2018-04-16 20:04:58 +02:00
4f3856adfb Retain old name for TLS client auth
I renamed `tlsAuth` to `tlsClientAuth` to better describe the fact that
this variable is used to enable TLS client authentication (as opposed to
server authentication) in c04d95f35.

However, changing the name breaks backwards compatibility for existing
installations using this feature and Grafana does not have a
standardised way of migrating changes in the schema:
https://github.com/grafana/grafana/pull/9377#issuecomment-333063543

For reasons of expediency given the severity of the bug (not verifying
TLS), keep the old name.
2017-10-11 08:18:04 +01:00
f6aa0e41e5 Return error if datasource TLS CA not parsed 2017-10-11 08:18:03 +01:00
5d312be419 Datasource HTTP settings: Add TLS skip verify
In c04d95f35 I changed the default for datasource HTTP requests so that
TLS is always verified.

This commit adds a checkbox to allow an admin to explicitly skip TLS
verification, for testing purposes.
2017-10-11 08:18:03 +01:00
43169e4302 Verify datasource TLS and split client auth and CA 2017-10-11 08:18:03 +01:00
e9e8ffc769 Merge pull request #9367 from bobrik/dualstack
Enable dualstack in every net.Dialer, fixes #9364
2017-09-28 11:15:49 +02:00
9a4ae30227 Enable dualstack in every net.Dialer, fixes #9364
Default transport enables it:

* https://github.com/golang/go/blob/d2826d3e06/src/net/http/transport.go#L42-L46

```
    DialContext: (&net.Dialer{
	Timeout:   30 * time.Second,
	KeepAlive: 30 * time.Second,
	DualStack: true,
    }).DialContext,
```

See also: https://github.com/golang/go/issues/15324
2017-09-27 22:29:48 -07:00
8175783f93 go fmt 2017-09-24 18:59:21 +02:00
cffbcb504d allow ssl renegotiation for datasources 2017-09-13 16:20:24 -07:00
2c7adccf12 Use cache for http.client in tsdb package. (#6833)
* datasource: move caching closer to datasource struct

* tsdb: use cached version of datasource http transport

closes #6825
2016-12-07 11:10:42 +01:00