49 Commits

Author SHA1 Message Date
2c7c2088d9 Logs panel: Add meta field to show total hits; add total hits to ElasticSearch plugin response (#104117)
* feat: Show total amount of hits in Elastic Search query

* Add test with multiple series.
2025-04-28 08:32:28 -05:00
f8c43d0bf3 Elasticsearch: Decouple backend from infra/tracing (#90528) 2024-07-19 09:26:10 +02:00
9caa8151d8 Elasticsearch: Decouple backend from infra/log (#90527) 2024-07-19 08:51:18 +02:00
494d169980 Elasticsearch: Fix legend for alerting, expressions and previously frontend queries (#84485)
* Elasticsearch: Fix legend for alerting, expressions and previously frontend queries

* Add comment

* Update comment
2024-03-18 18:01:33 +01:00
f2936d6695 Elasticsearch: Fix creating of legend so it is backward compatible with frontend produced frames (#81708)
* Elasticsearch: Fix creating of legend so it is backward compatible with frontend produced frames

* Update tests
2024-02-02 12:47:45 +01:00
759c088ac5 Elasticsearch: Fix showing of logs when __source is log message field (#80804)
Elasticsearch: Fix showing of logs whe __source is log message field
2024-01-19 10:28:08 +01:00
28f4c3ed44 Elasticsearch: Fix processing of raw_data with not-recognized time format (#78262)
* Elasticsearch: Fix non-standard time field in raw data queries

* Update snapshot tests

* Refactor
2023-11-20 12:01:41 +01:00
0128d0403f Tracing: Use tracing.InitializeTracerForTest (#75479) 2023-09-27 09:51:57 +02:00
4f0b31d21b Elasticsearch: Add tracing to data source (#74750)
* Elasticsearch: Add tracing do data source

* Fix tests

* Address feedback

* Update pkg/tsdb/elasticsearch/response_parser.go

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>

* Update pkg/tsdb/elasticsearch/response_parser.go

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>

* Track error across both spans

* Add span for decoding of response

* Fix test

* Update setting of errors + fix test

---------

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>
2023-09-18 10:49:12 +02:00
51391a762b Elasticsearch: Improve backend instrumentation of QueryData calls (#74172)
* Elasticsearch: Improve backend instrumentation of QueryData calls

* Add fromAlert

* Fix tests, move logger and context to struct

* Add instrumentation for processing response

* Move log for sending request closer to sending request

* Update

* Fix logging, improved messages, fix printing of queries

* Update log text

* Fix tests

* Fix lint

* Update logging to follow our guidelines

* Remove key-value pairs from logs that are going to be passed from instrumentation

* Update pkg/tsdb/elasticsearch/elasticsearch.go
2023-09-07 13:54:16 +02:00
025b2f3011 Chore: use any rather than interface{} (#74066) 2023-08-30 18:46:47 +03:00
4c9469fc9e Elastic: Add id field to Elastic responses to allow permalinking (#73382)
* add `id` field to elasticsearch

* add comment

* slightly better perf

* only add `id` to logs frames

* only add `id` for logs responses

* concat `index` and `id`

* change snapshot generation to false

* use better loop

* fix tests

* moved up
2023-08-18 11:39:58 +02:00
c1f6b91ea9 Elasticsearch: Fix multiple max depth flatten of multi-level objects (#70302) 2023-07-14 11:48:00 +02:00
dbbbc46351 Elasticsearch: Fix passing of limit and datalinks to logs data frame (#68554)
* Elasticsearch: Fix passing of limit and datalinks to logs data frame

* Update public/app/core/logsModel.ts

Co-authored-by: François Massot <francois.massot@gmail.com>

---------

Co-authored-by: François Massot <francois.massot@gmail.com>
2023-05-17 14:28:32 +02:00
95cf598423 Elasticsearch: Fix processing of logs with not-recognized time format (#67767)
* Elasticsearch: Fix parsing of invalid time

* Fix lint

* Add more test data to data.js

* Add tests

* Fix lint

* Update pkg/tsdb/elasticsearch/client/search_request.go
2023-05-04 19:33:00 +02:00
c308118fc0 Elasticsearch: Move response parsing tests to 1 file (#67288)
* Organize tests

* Organize

* Fix lint

* Fix lint
2023-04-26 18:21:04 +02:00
e2e243d1b6 Elasticsearch: Fix processing of duplicated metric types and field (#66973)
* Elasticsearch: Fix processing of duplicated metric types and field

* Fix lint

* Fix linting
2023-04-24 17:26:04 +02:00
3e4eb9c93e Elasticsearch: Fix legend generated on backend (#66039)
* Elasticsearch: Fix legend

* Remove comments

* fix adding name to config

* Fix lint
2023-04-11 13:29:38 +02:00
ae0f9fc44f Elasticsearch: Fix processing of response with multiple group by for alerting (#65165)
* Elasticsearch: Fix processing of response with multiple group by on backend

* Remove unused param from createFields

* Switch comments
2023-03-22 18:53:09 +02:00
0740d4e911 Elasticsearch: Fix top metrics with terms in backend (#65063)
* Elasticsearch: Fix top metrics with terms in backend

* Refactor to remove cyclomatic complexity
2023-03-22 09:36:43 +01:00
bd0fa79542 Elasticsearch: Add log highlight processing on backend (#63924)
* Elasticsearch: Add highlight for logs

* Fix running of queries trough backend only when shouldRunTroughBackend

* Fix incorrect json in test string

* Address feedback

* Keep order of words same
2023-03-09 13:34:53 +01:00
aa123e0d50 Elasticsearch: Fix where name of frame is set (#64233)
* Elasticsearch: Use displayName field for naming

* Change solution to frame.Name to be backward compatible

* Fix snapshot tests

* Use Time and Value for time and value fields

* Use variables from grafana-plugin-sdk-go for name
2023-03-08 13:32:01 +01:00
3bd7217aa2 Elasticsearch: Remove _source field when processing raw data on backend (#64119)
* Elasticsearch: Remove _source field when processing raw data on backend

* Update snapshot test
2023-03-07 13:56:34 +01:00
adda7819e9 Elasticsearch: Implement processing of raw document query results in backend (#63932) 2023-03-02 09:50:54 +01:00
d258c8ef8a Elasticsearch: Implement processing of logs query results in backend (#63647)
* Elasticsearch: Add processing of logs query to backend

* Add and fix tests

* Add snapshot tests

* Fix test in ES client

* Small updates, remove redundant logic

* Refactor setPreferredVisType to improve readability
2023-03-01 05:50:56 -05:00
89b3663a23 Elasticsearch: Add processing for raw data to backend (#63208)
* WIP

* WIP

* Refactor

* Add tests

* Cleanup

* Fix whitespace

* Fix test and lint

* In snapshot tests update counter to be number

* Add boolean value for snapshot testing

* Update pkg/tsdb/elasticsearch/response_parser.go

Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>

* Update pkg/tsdb/elasticsearch/response_parser.go

Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>

* Use generic to reuse logic when creating fields

* Use nullable fields

* Fix lint

* WIP (#63272)

wip

* Fix snapshot test after we changed field types to nullable

---------

Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>
2023-02-22 13:28:43 +01:00
d9fd807375 Elasticsearch: Fix consistent label order in alerting (#62497)
elasticsearch: backend: sort label-values
2023-02-02 09:03:18 +01:00
0bf4093005 Chore: Fix goimports grouping (#62428)
* fix goimports order

* fix goimports order
2023-01-30 08:50:27 +00:00
57425234c9 elastic: backend: removed unused time-field (#61362)
* elastic: backend: removed unused time-field

* improve comment formatting

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>
2023-01-12 15:52:32 +00:00
12c4098bcd elastic: backend: handle naming bucket_scripts (#60460) 2023-01-04 15:26:57 +01:00
d332dab3ec Elasticsearch: Refactor parse query (#60440)
* Refactor parse query to functions

* Move parsing to new file

* Create empty result variable and use it when returning early

* Fix linting

* Revert "Create empty result variable and use it when returning early"

This reverts commit 36a503f66e52f8213c673972774329a963a78100.
2022-12-16 16:30:36 +01:00
5633843f76 Elasticsearch: backend: do not set name for time-fields (#60169)
elastic: backend: improve field-naming
2022-12-13 13:19:03 +01:00
3172d8dd9b elastic: response parser: simplify code (#59782)
elastic: simplify code
2022-12-12 16:00:15 +01:00
b8b2de1ac4 elastic: remove unused debug-code (#59712)
* elastic: remove unused debug-code

* removed unused structs
2022-12-05 04:21:15 -05:00
7996a1c8ad Chore: Update grafana-plugin-sdk-go to v0.142.0 (#58115)
* Chore: Update grafana-plugin-sdk-go to v0.142.0

* Update tests and golden files for 207 status code

* Chore: Move update flag definition at the top in response_parser_test.go

* retrigger

Co-authored-by: Will Browne <will.browne@grafana.com>
2022-11-03 18:34:27 +01:00
0491c19712 elastic: backend-mode: fix trimEdges functionality (#56985) 2022-10-19 10:40:42 +02:00
e4f2006cce Elasticsearch: Fix calculation of trimEdges (#56148)
* Elasticsearch: Fix calculation of trimEdges

When a value of trimEdges is set greater than 1 we need to drop both the
first and last sample of the data from the response.

* Elasticsearch: Fix reading trimEdges from the query settings

Currently the trimEdges property in the JSON panel is stored as a string
and not directly as a number.

This caused that the reading of the value failed in the go backend
because the simplejson.Int() method doesn't properly handle this case.
This failure when decoding the value goes unnoticed because of the early
return causing the trimEdges configuration to be ignored.

* Refactor castToInt to also return an error

Add a new test case that sets the `trimEdges` property as a quoted
number.
2022-10-14 12:53:12 +02:00
12320dda3c Elasticsearch: Fix metric names for alert queries (#38546)
* Export fieldName to var

* Add empty string as frame name
2021-08-26 09:20:38 +02:00
0df1b33d71 Plugins: Migrate Elasticsearch to backend plugin SDK (#36132)
* Migrate Elasticsearch to backend plugin SDK

* Fix linting

* Move away from Convey!

* Rebase commit

* Small logger fix

* Fixes according to reviewer's comments

* Fixes according to reviewer's comments

* Fixes according to reviewer's comments

* More cleanup

* Move things around - small refactoring

* Fix typo

* Update calculator - add tests

* Fixes according to reviewer's comments
2021-07-15 16:45:59 +02:00
0d08881876 Elasticsearch: Convert Timeseries and Tables to Dataframes (#34710)
* Rebase (broken tests)

* Removed tables - refactored processAggregationDocs func

* Tests cleanup

* Nit - add space in percentile legend titles

* Fix labels naming - use metricAggType map

* Fix bug which appended same fields over and over again

* Replace test with dataframes

* Fix tests

* Add nolint:gocyclo - will need refactoring

* Move frames tags assignment

* Add failing test fo when ES server is booting up

* Revert "Add failing test fo when ES server is booting up"

This reverts commit fd14a1fd5e181e475f9c5d6a52bd48496f24b822.

Co-authored-by: Elfo404 <me@giordanoricci.com>
2021-06-18 13:26:19 +03:00
f580c9149c Elasticsearch: Add Top Metrics Aggregation and X-Pack support (#33041)
* Elasticsearch: Add Top Metrics Aggregation

* Adding support for non-timeseries visualizations

* removing console.logs

* restoring loadOptions type

* Honor xpack setting

* Adding test for elastic_response

* adding test for query builder

* Adding support of alerting

* Fixing separator spelling

* Fixing linting issues

* attempting to reduce cyclomatic complexity

* Adding elastic77 Docker block

* Update public/app/plugins/datasource/elasticsearch/components/QueryEditor/MetricAggregationsEditor/MetricEditor.test.tsx

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>

* refactoring MetricsEditor tests

* Fixing typo

* Change getFields type & move TopMetrics to a separate component

* Fix SegmentAsync styles in TopMetrics Settings

* Fix field types for TopMetrics

* WIP

* Refactoring client side to support multiple top metrics

* Adding tests and finishing go implimentation

* removing fmt lib from debugging

* fixing tests

* reducing the cyclomatic complexity

* Update public/app/plugins/datasource/elasticsearch/elastic_response.ts

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>

* Update public/app/plugins/datasource/elasticsearch/hooks/useFields.ts

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>

* Checking for possible nil value

* Fixing types

* fix fake-data-gen param

* fix useFields hook

* Removing aggregateBy and size

* Fixing go tests

* Fixing TS tests

* fixing tests

* Fixes

* Remove date from top_metrics fields

* Restore previous formatting

* Update pkg/tsdb/elasticsearch/client/models.go

Co-authored-by: Dimitris Sotirakis <dimitrios.sotirakis@grafana.com>

* Update pkg/tsdb/elasticsearch/client/models.go

Co-authored-by: Dimitris Sotirakis <dimitrios.sotirakis@grafana.com>

* Fix code review comments on processTopMetricValue

* Remove underscore from variable names

* Remove intermediate array definition

* Refactor test to use testify

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>
Co-authored-by: Elfo404 <me@giordanoricci.com>
Co-authored-by: Dimitris Sotirakis <dimitrios.sotirakis@grafana.com>
2021-06-04 11:07:59 +01:00
b79e61656a Introduce TSDB service (#31520)
* Introduce TSDB service

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>
2021-03-08 07:02:49 +01:00
be961c5466 Elasticsearch: Fix using multiple bucket script aggregations when only grouping by terms (#24064)
* In the parser when it was configurate more bucket it put the formula in name column

* Tests

* Tests

* Tests

* Tests

* Tests

* Tests

* Process/Tests - backend (Go)

* Update pkg/tsdb/elasticsearch/response_parser.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* reverse

* Update pkg/tsdb/elasticsearch/response_parser.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2020-07-09 15:21:19 +02:00
5070f7a75b Chore: Start harmonizing linting with plugin SDK (#25854)
* Chore: Harmonize linting with plugin SDK

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Chore: Fix linting issues

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-06-29 14:08:32 +02:00
5713048f48 Alerting: Improve alert rule testing (#16286)
* tsdb: add support for setting debug flag of tsdb query

* alerting: adds debug flag in eval context

Debug flag is set when testing an alert rule and this debug
flag is used to return more debug information in test aler rule
response. This debug flag is also provided to tsdb queries so
datasources can optionally add support for returning additional
debug data

* alerting: improve test alert rule ui

Adds buttons for expand/collapse json and copy json to clipboard,
very similar to how the query inspector works.

* elasticsearch: implement support for tsdb query debug flag

* elasticsearch: embedding client response in struct

* alerting: return proper query model when testing rule
2019-06-25 08:52:17 +02:00
b45f72a140 elasticsearch: support bucket script pipeline aggregations 2018-12-20 01:24:36 +01:00
4840adff00 elasticsearch: refactor query handling and use new es simple client
Removes moment dependency.
Adds response parser tests (based on frontend tests).
Adds time series query tests (based on frontend tests).
Fixes various issues related to response parsing and building search request queries.
Added support for extended stats metrics and geo hash grid aggregations.
2018-05-23 15:09:58 +02:00
77400cef08 elasticsearch: refactor and cleanup
Move time series query logic to specific file.
Remove model parser and move to time series query file, adds parser test.
2018-05-23 14:36:41 +02:00
4050fce220 add response_parser test 2018-03-28 12:35:05 +08:00