65 Commits

Author SHA1 Message Date
4a7a3f9e2a ElasticSearch: Fallback parse total hits as int (#105615)
* Attempt int parsing

* Update total hits logic

- Do not allow the error to fail the request
- Add some comments

* Mark log as debug
2025-05-19 18:08:38 +01:00
2c7c2088d9 Logs panel: Add meta field to show total hits; add total hits to ElasticSearch plugin response (#104117)
* feat: Show total amount of hits in Elastic Search query

* Add test with multiple series.
2025-04-28 08:32:28 -05:00
27795ff7bf ElasticSearch: Improve index pattern error messaging and docs (#103899) 2025-04-14 15:55:33 +02:00
66279081c6 Elasticsearch: Invalid response JSON parsing error should be downstream (#101506)
* Elasticsearch: Invalid response JSON parsing error should be downstream

* Add test

* Update test
2025-03-03 18:14:43 +01:00
30aa676724 Elasticsearch: Invalid URL and unsupported protocol should be downstream errors (#100886)
* Invalid URL and unsupported protocol should be downstream errors

* Fix lint

* Change from errors.Is to errors.As
2025-02-20 10:04:48 +01:00
7092fd269d Elasticsearch: Remove frontend testDatasource method (#99894) 2025-02-10 18:33:36 +01:00
bab55a4cb8 Elasticsearch: Process ES multisearch JSON response by streaming (#93689)
Co-authored-by: Isabella Siu <isabella.siu@grafana.com>
2025-01-15 12:05:54 -05:00
7de65b39b5 Elasticsearch and Cloudwatch: Replace deprecated experimental/errorsource (#97363)
* Update elasticsearch

* Update azure monitor

* Update test

* Revert "Update azure monitor"

This reverts commit 21b53845b85c3a43d679d729c33879fdc3e9f027.

* Update cloudwatch
2024-12-12 10:12:03 +01:00
6000d6a1ad ElasticSearch: Neaten up vars/consts (#91989)
ElasticSearch: neaten up vars/consts
2024-08-16 09:42:39 -04:00
f8c43d0bf3 Elasticsearch: Decouple backend from infra/tracing (#90528) 2024-07-19 09:26:10 +02:00
9caa8151d8 Elasticsearch: Decouple backend from infra/log (#90527) 2024-07-19 08:51:18 +02:00
336b6dafb6 Elasticsearch: Fix setting of default maxConcurrentShardRequests (#87703)
* Elasticsearch: Fix setting of default maxConcurrentShardRequests

* Add float test scenario

* Add comment
2024-05-13 16:39:37 +03:00
77b1c97482 Elasticsearch: Remove xpack button and make includeFrozen not dependant on it (#84734) 2024-03-20 09:59:26 +01:00
2a1a5145d0 Elasticsearch: Fix using of individual query time ranges when querying (#84201)
* WIP - proof of concept

* Update pkg/tsdb/elasticsearch/client/client.go

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>

* update and add test

* lint

* Fix lint

* Bring back logging when creating client

---------

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>
2024-03-13 12:49:35 +02:00
7f7ab32444 Elasticsearch: Add error source to logs (#82901) 2024-02-16 16:28:46 +01:00
Sam
0cf6b94fa2 tsdb/elasticsearch: remove TimeInterval field from tsdb/elasticsearch client (#78264)
remove TimeInterval field from tsdb/elasticsearch client #78187

Co-authored-by: Sam <samucap@users.noreply.github.com>
2023-11-16 14:57:21 +01:00
e4c1a7a141 Tracing: Standardize on otel tracing (#75528) 2023-10-03 14:54:20 +02:00
4f0b31d21b Elasticsearch: Add tracing to data source (#74750)
* Elasticsearch: Add tracing do data source

* Fix tests

* Address feedback

* Update pkg/tsdb/elasticsearch/response_parser.go

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>

* Update pkg/tsdb/elasticsearch/response_parser.go

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>

* Track error across both spans

* Add span for decoding of response

* Fix test

* Update setting of errors + fix test

---------

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>
2023-09-18 10:49:12 +02:00
49bd93aa2b Elasticsearch: Adjust logging to follow convention (#74554)
* Elasticsearch: Adjust naming in logging according to convention

* Log response parsing per response

* Update

* Fix logging of errors when no response

* Add path to error loggigng

* Update pkg/tsdb/elasticsearch/response_parser.go
2023-09-07 18:15:24 +02:00
51391a762b Elasticsearch: Improve backend instrumentation of QueryData calls (#74172)
* Elasticsearch: Improve backend instrumentation of QueryData calls

* Add fromAlert

* Fix tests, move logger and context to struct

* Add instrumentation for processing response

* Move log for sending request closer to sending request

* Update

* Fix logging, improved messages, fix printing of queries

* Update log text

* Fix tests

* Fix lint

* Update logging to follow our guidelines

* Remove key-value pairs from logs that are going to be passed from instrumentation

* Update pkg/tsdb/elasticsearch/elasticsearch.go
2023-09-07 13:54:16 +02:00
025b2f3011 Chore: use any rather than interface{} (#74066) 2023-08-30 18:46:47 +03:00
e17540bdcd Elasticsearch: Fix using multiple indexes with comma separated string (#71284)
* Revert "Elasticsearch: Use array of strings as index in backend queries (#67276)"

This reverts commit d0ced39847c94a76047c2e79824f1ccb262e3a33.

* updated tests
2023-07-11 09:47:16 +02:00
d0ced39847 Elasticsearch: Use array of strings as index in backend queries (#67276)
Elasticsearch: Use array of strings as indice in backend queries
2023-04-26 16:07:15 +02:00
0cff917f2a Elasticsearch: Removed reference to obsolete esVersion value (#65415)
* elastic: removed reference to obsolete esVersion value

* removed unused code

* cleaned up tests
2023-03-28 17:04:56 +03:00
e8bb894a02 Elasticsearch: Remove GetMinInterval method that is not used anymore (#65203) 2023-03-23 09:36:37 +01:00
d258c8ef8a Elasticsearch: Implement processing of logs query results in backend (#63647)
* Elasticsearch: Add processing of logs query to backend

* Add and fix tests

* Add snapshot tests

* Fix test in ES client

* Small updates, remove redundant logic

* Refactor setPreferredVisType to improve readability
2023-03-01 05:50:56 -05:00
772e8cbf60 Elasticsearch: Use interval provided by data request in backend (#60480)
* Elasticsearch: Remove interval caulation and use interval provises by grafana

* Remove redundant code

* Adjust snapshot tests

* Update test

* Fix lint
2023-01-05 12:26:27 +01:00
b8b2de1ac4 elastic: remove unused debug-code (#59712)
* elastic: remove unused debug-code

* removed unused structs
2022-12-05 04:21:15 -05:00
d9c40ca41e Chore: Update sqleng, elasticsearch, tempo and opentsdb plugins to support contextual logs. (#57777)
* make sql engine use pick log context for logs
* update tempo to get log context
* update opentsdb to use log context
* update es client to use log context
2022-11-02 10:03:50 -04:00
2efd7fa481 Elasticsearch: Reuse http client in the backend (#55172)
* elastic: backend: reuse http client

* fixed lint error
2022-09-26 14:27:46 +02:00
35253a909d Elasticsearch: remove already-disabled code (#52932)
* elastic: removed unused code from go-part

* elastic: removed unused code from js-part
2022-08-22 16:25:20 +02:00
Jo
062d255124 Handle ioutil deprecations (#53526)
* replace ioutil.ReadFile -> os.ReadFile

* replace ioutil.ReadAll -> io.ReadAll

* replace ioutil.TempFile -> os.CreateTemp

* replace ioutil.NopCloser -> io.NopCloser

* replace ioutil.WriteFile -> os.WriteFile

* replace ioutil.TempDir -> os.MkdirTemp

* replace ioutil.Discard -> io.Discard
2022-08-10 15:37:51 +02:00
8f6877e12a chore: remove all remaining uses of golang.org/x/net/context (#47564)
* chore: remove all remaining uses of  golang.org/x/net/context

This PR finishes the work started in #47532, replacing all calls to that package with the stdlib context and using http.NewRequestWithContext to include the context where necessary.

Bonus: small formatting fixes to goimports in these files.

closes #44178

* tweak: use context.Background in favor of TODO for tests
2022-04-11 14:20:10 -04:00
76d88a6cec change log level to debug (#41102) 2021-11-01 09:00:00 +00:00
c989f0fbbe SQL: Migrate to use SDK contracts (#36635)
* convert SQLs to use sdk contracts

* make draft

* postgres

* intermedia

* get datasourceinfo filled at the beginning of the service

* move the interval into package because of cyclict  import and fix all postgres tests

* fix mysql test

* fix mssql

* fix the test for pr https://github.com/grafana/grafana/issues/35839

* fix some issue about intervalv2 package

* update sql test

* wire migration for SQLs

* add sqls to the background process

* make it register instead of register and start

* revert formatting

* fix tests

* fix linter

* remove integration test

* Postgres test fix

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-09-07 09:35:37 +02:00
0df1b33d71 Plugins: Migrate Elasticsearch to backend plugin SDK (#36132)
* Migrate Elasticsearch to backend plugin SDK

* Fix linting

* Move away from Convey!

* Rebase commit

* Small logger fix

* Fixes according to reviewer's comments

* Fixes according to reviewer's comments

* Fixes according to reviewer's comments

* More cleanup

* Move things around - small refactoring

* Fix typo

* Update calculator - add tests

* Fixes according to reviewer's comments
2021-07-15 16:45:59 +02:00
f6b83a4f47 Elasticsearch: Add frozen indices search support (#36018)
* Revert "Revert "Elasticsearch: add frozen indices search support (#27472)" (#27726)"

This reverts commit 4c7131425ba048c648435410f2153d4d85f20304.

* Make label width a bit more consistent

* Add documentation for X-Pack & Frozen Indices support in Elasticsearch

* Change UI & docs casing

* create default empty dataframe

* Remove backticks and simplify regex

* small doc improvement
2021-07-15 15:52:02 +02:00
f580c9149c Elasticsearch: Add Top Metrics Aggregation and X-Pack support (#33041)
* Elasticsearch: Add Top Metrics Aggregation

* Adding support for non-timeseries visualizations

* removing console.logs

* restoring loadOptions type

* Honor xpack setting

* Adding test for elastic_response

* adding test for query builder

* Adding support of alerting

* Fixing separator spelling

* Fixing linting issues

* attempting to reduce cyclomatic complexity

* Adding elastic77 Docker block

* Update public/app/plugins/datasource/elasticsearch/components/QueryEditor/MetricAggregationsEditor/MetricEditor.test.tsx

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>

* refactoring MetricsEditor tests

* Fixing typo

* Change getFields type & move TopMetrics to a separate component

* Fix SegmentAsync styles in TopMetrics Settings

* Fix field types for TopMetrics

* WIP

* Refactoring client side to support multiple top metrics

* Adding tests and finishing go implimentation

* removing fmt lib from debugging

* fixing tests

* reducing the cyclomatic complexity

* Update public/app/plugins/datasource/elasticsearch/elastic_response.ts

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>

* Update public/app/plugins/datasource/elasticsearch/hooks/useFields.ts

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>

* Checking for possible nil value

* Fixing types

* fix fake-data-gen param

* fix useFields hook

* Removing aggregateBy and size

* Fixing go tests

* Fixing TS tests

* fixing tests

* Fixes

* Remove date from top_metrics fields

* Restore previous formatting

* Update pkg/tsdb/elasticsearch/client/models.go

Co-authored-by: Dimitris Sotirakis <dimitrios.sotirakis@grafana.com>

* Update pkg/tsdb/elasticsearch/client/models.go

Co-authored-by: Dimitris Sotirakis <dimitrios.sotirakis@grafana.com>

* Fix code review comments on processTopMetricValue

* Remove underscore from variable names

* Remove intermediate array definition

* Refactor test to use testify

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>
Co-authored-by: Elfo404 <me@giordanoricci.com>
Co-authored-by: Dimitris Sotirakis <dimitrios.sotirakis@grafana.com>
2021-06-04 11:07:59 +01:00
530072fce7 Elasticsearch: fix NewClient not passing httpClientProvider to client impl (#34539)
* Update client.go

fix missing "httpClientProvider" in NewClient

* Update client.go
2021-05-24 14:45:29 +01:00
348e76fc8e Datasource: Shared HTTP client provider for core backend data sources and any data source using the data source proxy (#33439)
Uses new httpclient package from grafana-plugin-sdk-go introduced 
via grafana/grafana-plugin-sdk-go#328. 
Replaces the GetHTTPClient, GetTransport, GetTLSConfig methods defined 
on DataSource model.
Longer-term the goal is to migrate core HTTP backend data sources to use the 
SDK contracts and using httpclient.Provider for creating HTTP clients and such.

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-05-19 23:53:41 +02:00
e98a8bd11b Elasticsearch: use semver strings to identify ES version (#33646)
* Elasticsearch: use proper semver strings to identify ES version

* Update BE & tests

* refactor BE tests

* refactor isValidOption check

* update test

* Update pkg/tsdb/elasticsearch/client/client.go

Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>

* Update pkg/tsdb/elasticsearch/client/search_request_test.go

Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>

* Remove leftover FIXME comment

* Add new test cases for new version format

* Docs: add documentation about version dropdown

* Update docs/sources/datasources/elasticsearch.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/elasticsearch.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/elasticsearch.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update provisioning documentation

Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
2021-05-11 09:44:00 +01:00
f7b408f99f Elasticsearch: use application/x-ndjson content type for multisearch requests (#32282) 2021-03-29 23:41:45 +01:00
b79e61656a Introduce TSDB service (#31520)
* Introduce TSDB service

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>
2021-03-08 07:02:49 +01:00
c2cad26ca9 Chore: Disable default golangci-lint filter (#29751)
* Disable default golangci-lint filter

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Chore: Fix linter warnings

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-12-15 09:32:06 +01:00
4c7131425b Revert "Elasticsearch: add frozen indices search support (#27472)" (#27726)
This reverts commit 61a3160c34f659be848d67a10bd9e0e5be6e1b58.
2020-09-23 12:24:46 +01:00
a5d9196a53 Chore/fix lint issues (#27704)
* Chore: Fix linting issues

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-09-22 16:22:19 +02:00
61a3160c34 Elasticsearch: add frozen indices search support (#27472)
* Elasticsearch: Add frozen indices support for alerting
2020-09-18 18:27:25 +01:00
5070f7a75b Chore: Start harmonizing linting with plugin SDK (#25854)
* Chore: Harmonize linting with plugin SDK

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Chore: Fix linting issues

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-06-29 14:08:32 +02:00
58cefe73ee Datasources: Handle URL parsing error (#25742)
Adds handling of error returned from URL parsing.

Fixes #25714
2020-06-22 16:34:40 +02:00
5713048f48 Alerting: Improve alert rule testing (#16286)
* tsdb: add support for setting debug flag of tsdb query

* alerting: adds debug flag in eval context

Debug flag is set when testing an alert rule and this debug
flag is used to return more debug information in test aler rule
response. This debug flag is also provided to tsdb queries so
datasources can optionally add support for returning additional
debug data

* alerting: improve test alert rule ui

Adds buttons for expand/collapse json and copy json to clipboard,
very similar to how the query inspector works.

* elasticsearch: implement support for tsdb query debug flag

* elasticsearch: embedding client response in struct

* alerting: return proper query model when testing rule
2019-06-25 08:52:17 +02:00