42 Commits

Author SHA1 Message Date
57425234c9 elastic: backend: removed unused time-field (#61362)
* elastic: backend: removed unused time-field

* improve comment formatting

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>
2023-01-12 15:52:32 +00:00
30424a8618 Elasticsearch: Return error if invalid query (#61044)
* Elasticsearch: Return error if invalid query

* Add test

* Fix query in test
2023-01-06 11:14:17 +01:00
772e8cbf60 Elasticsearch: Use interval provided by data request in backend (#60480)
* Elasticsearch: Remove interval caulation and use interval provises by grafana

* Remove redundant code

* Adjust snapshot tests

* Update test

* Fix lint
2023-01-05 12:26:27 +01:00
a1ef3d77f2 Elasticsearch: Use constants for default values and types (#60673) 2022-12-22 15:08:15 +01:00
09bb4423d2 Elasticsearch: Fix pipeline aggregation queries in backend to pass frontend tests (#60337)
* Elasticsearch: Fix pipeline aggregation queries in backend

* Update

* Update lint

* Update pkg/tsdb/elasticsearch/time_series_query.go

* Fix lint

* Fix merge
2022-12-16 17:45:43 +01:00
d332dab3ec Elasticsearch: Refactor parse query (#60440)
* Refactor parse query to functions

* Move parsing to new file

* Create empty result variable and use it when returning early

* Fix linting

* Revert "Create empty result variable and use it when returning early"

This reverts commit 36a503f66e52f8213c673972774329a963a78100.
2022-12-16 16:30:36 +01:00
70e34e7f32 Elasticsearch: Fix defaulting to time field if no field is specified in aggregation (#60238)
* Elasticsearch: Add test for using timeField from datasource when not specified in query

* Add default to timefield

* Merge fix
2022-12-14 14:19:03 +01:00
d3ef86bd90 Elasticsearch: Add query building for log queries (#60182)
* Elasticsearch: Fix ordering in raw_document and add logic for raw_data

* Add comments

* Fix raw data request to use correct timefield

* Fix linting

* Add raw data as metric type

* Fix linting

* Elasticsearch: Add defaults for log query

* Add higlight

* Fix lint

* Add snapshot test

* Implement correct query for logs

* Update

* Adjust naming and comments

* Fix lint

* Remove ifs
2022-12-14 13:56:09 +01:00
0973501233 Elasticsearch: Add query building for raw data and raw document queries to backend (#59741)
* Elasticsearch: Fix ordering in raw_document and add logic for raw_data

* Add comments

* Fix raw data request to use correct timefield

* Fix linting

* Add raw data as metric type

* Fix linting

* Hopefully fix lint
2022-12-13 13:11:44 +01:00
13acddd605 Elasticsearch: Fix adding correct min_doc_count to all aggregation types in backend implementation (#59713)
* Elasticsearch: Fix adding correct min_doc_count to all aggregation types in backend implementation

* Update snapshot tests

* Fix linting
2022-12-07 09:47:31 +01:00
e356a56741 Elasticsearch: Fix removing of empty settings from query in backend implementation (#59704)
* Elasticsearch: Fix removing of empty settings from query in backend implementation

* Update

* Update

* Update pkg/tsdb/elasticsearch/time_series_query.go

Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>

Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>
2022-12-05 16:06:55 +01:00
b8b2de1ac4 elastic: remove unused debug-code (#59712)
* elastic: remove unused debug-code

* removed unused structs
2022-12-05 04:21:15 -05:00
a2ed586576 Elasticsearch: Migrate frontend query builder tests to backend (#59578)
* WIP: Migrated first batch of tests

* Migrate more tests

* Migrate rest of the tests

* Update comments

* Update

* Update

* Remove newline
2022-12-01 12:09:12 +01:00
a58edc9f5e elastic: always use millisecond-intervals for alerting (#54157) 2022-08-26 14:40:28 +02:00
35253a909d Elasticsearch: remove already-disabled code (#52932)
* elastic: removed unused code from go-part

* elastic: removed unused code from js-part
2022-08-22 16:25:20 +02:00
a2eb4e85e5 elasticsearch: always use fixed_interval (#50297) 2022-06-13 10:28:29 +02:00
9fb8339f87 Elastic: Allow using long/int as date field for alerts (#44027)
* Use integers for time range filter

Previously it was passed as a string which is automatically converted by Elastic to a number only if the field type is "date". For other types (e.g. "long") such conversion doesn't work. In theory "date" could be passed as a formatted string but we don't use it this way and always pass it as a number so it is safe to always pass numbers, not strings.

* Fix time_series_query_test

* Retrigger build
2022-01-17 15:45:09 +01:00
1b99d88337 Elasticsearch: Add support for Elasticsearch 8.0 (Beta) (#41729)
* use fixed_interval in date_histogram

* Add 8.0 to available versions in datasource settings

* Remove moving_avg from available metric aggregations

* Add ES8 devenv

* Update public/app/plugins/datasource/elasticsearch/components/QueryEditor/MetricAggregationsEditor/utils.ts

Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>

* Add FE tests

* Add BE test

* fix FE test

* fix BE test

Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>
2021-11-17 20:30:53 +01:00
baab021fec Chore: Refactor usage of legacy data contracts (#41218)
Refactor usage of legacy data contracts. Moves legacy data contracts 
to pkg/tsdb/legacydata package.
Refactor pkg/expr to be a proper service/dependency that can be provided 
to wire to remove some unneeded dependencies to SSE in ngalert and other places.
Refactor pkg/expr to not use the legacydata,RequestHandler and use 
backend.QueryDataHandler instead.
2021-11-10 11:52:16 +01:00
7b7c193551 Elasticsearch: Add time zone setting to Date Histogram aggregation (#40882)
* TimeZonePicker: Allow specifying internal timezones to display

* Elasticsearch: Add time zone setting to Date Histogram aggregation

* rename time_zone to timeZone

* Add tests

* fix typo

* Update packages/grafana-data/src/datetime/timezones.ts

Co-authored-by: Chris Cowan <chris@chriscowan.us>

Co-authored-by: Chris Cowan <chris@chriscowan.us>
2021-11-02 16:18:39 +00:00
c989f0fbbe SQL: Migrate to use SDK contracts (#36635)
* convert SQLs to use sdk contracts

* make draft

* postgres

* intermedia

* get datasourceinfo filled at the beginning of the service

* move the interval into package because of cyclict  import and fix all postgres tests

* fix mysql test

* fix mssql

* fix the test for pr https://github.com/grafana/grafana/issues/35839

* fix some issue about intervalv2 package

* update sql test

* wire migration for SQLs

* add sqls to the background process

* make it register instead of register and start

* revert formatting

* fix tests

* fix linter

* remove integration test

* Postgres test fix

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-09-07 09:35:37 +02:00
0df1b33d71 Plugins: Migrate Elasticsearch to backend plugin SDK (#36132)
* Migrate Elasticsearch to backend plugin SDK

* Fix linting

* Move away from Convey!

* Rebase commit

* Small logger fix

* Fixes according to reviewer's comments

* Fixes according to reviewer's comments

* Fixes according to reviewer's comments

* More cleanup

* Move things around - small refactoring

* Fix typo

* Update calculator - add tests

* Fixes according to reviewer's comments
2021-07-15 16:45:59 +02:00
7a304223e6 Elasticsearch: Fix min_doc_count value when alerting (#35254)
* Elasticsearch: Fix min_doc_count value when alerting

* Add tests
2021-06-04 16:42:00 +02:00
e2e78f14d2 Elasticsearch: fix flaky test (#34517) 2021-05-20 17:13:48 +02:00
8ec87250c1 Elasticsearch: Fix handling of inline scripts in different ES versions (#34070)
* Devenv: add block for es 5.0, provisioned datasource & dashboard

* Trasnsform script property based on running ES version

* Handle different scripts format in BE
2021-05-14 11:50:15 +01:00
e98a8bd11b Elasticsearch: use semver strings to identify ES version (#33646)
* Elasticsearch: use proper semver strings to identify ES version

* Update BE & tests

* refactor BE tests

* refactor isValidOption check

* update test

* Update pkg/tsdb/elasticsearch/client/client.go

Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>

* Update pkg/tsdb/elasticsearch/client/search_request_test.go

Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>

* Remove leftover FIXME comment

* Add new test cases for new version format

* Docs: add documentation about version dropdown

* Update docs/sources/datasources/elasticsearch.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/elasticsearch.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/elasticsearch.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update provisioning documentation

Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
2021-05-11 09:44:00 +01:00
c88af6e221 Elasticsearch: Add generic support for template variables (#32762)
* Elasticsearch: Add generic support for template variables

* format MovingAverage settings as numbers

* Move formatting logic to query builder & forma serial_diff settings as numbers

* modify presence check

* add todo

* minor fixes

* transform string values to numbers

* Move casting logic

* Slightly cleaner implementation

* Add BE tests

* Leverage elastic validation when string doesn't resolve to a numeric value

* move newly introduced test to testify

* add FE query_builder tests

* check error

* Parse values to float instead of int

* Fix tests & ParseFloat bit size
2021-04-26 16:54:23 +01:00
7627b55ef4 TSDB: add deprecation comments to many tsdb structs (#33281) 2021-04-23 03:03:11 +02:00
b79e61656a Introduce TSDB service (#31520)
* Introduce TSDB service

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>
2021-03-08 07:02:49 +01:00
5088e2044a Elasticsearch: Support extended stats and percentiles in terms order by (#28910)
Adds support to the terms aggregation for ordering by percentiles and extended stats. 

Closes #5148

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-01-15 12:10:16 +01:00
3d7748d9f8 Elasticsearch: Use minimum interval for alerts (#30049)
The current backend code doesn't honor the minimum interval set in the UI for alerts using 
the Elasticsearch data source. This means that the data the alerts are triggering against 
will never match the data in the visualization if auto is used in the date histogram as interval. 
This fixes the problem to make sure that date histogram auto interval is set according to 
min interval set in UI for the query or fallback to data source min interval setting.

Fixes #22082

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-01-14 10:48:59 +01:00
cf1c01dd8b Elasticsearch: Add Support for Serial Differencing Pipeline Aggregation (#28618)
* Elasticsearch: Add support for serial diff pipeline aggregation

* Removing settings transsforms

* Removing unused deps

* removing unused dep

* Fixing type in test

* Adding backend support for serial_diff
2021-01-08 11:52:23 +00:00
5070f7a75b Chore: Start harmonizing linting with plugin SDK (#25854)
* Chore: Harmonize linting with plugin SDK

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Chore: Fix linting issues

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-06-29 14:08:32 +02:00
51d1261a4b Elasticsearch: Adds cumulative sum aggregation support (#24820)
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2020-06-01 23:55:15 +02:00
5713048f48 Alerting: Improve alert rule testing (#16286)
* tsdb: add support for setting debug flag of tsdb query

* alerting: adds debug flag in eval context

Debug flag is set when testing an alert rule and this debug
flag is used to return more debug information in test aler rule
response. This debug flag is also provided to tsdb queries so
datasources can optionally add support for returning additional
debug data

* alerting: improve test alert rule ui

Adds buttons for expand/collapse json and copy json to clipboard,
very similar to how the query inspector works.

* elasticsearch: implement support for tsdb query debug flag

* elasticsearch: embedding client response in struct

* alerting: return proper query model when testing rule
2019-06-25 08:52:17 +02:00
b45f72a140 elasticsearch: support bucket script pipeline aggregations 2018-12-20 01:24:36 +01:00
9a8ad70013 fix pipeline aggregations on doc count 2018-11-15 19:06:47 +01:00
5803bfd2c7 fix terms agg order deprecation warning on es 6+ 2018-11-02 17:53:47 +01:00
18dfdc4f0d add test for es alert when group by has no limit 2018-10-11 16:04:57 +08:00
dcac63936b elasticsearch: minor refactor
Handle all replacements if interval template variables in the client.
Fix issue with client and different versions.
Adds better tests of the client
2018-05-31 19:02:20 +02:00
4840adff00 elasticsearch: refactor query handling and use new es simple client
Removes moment dependency.
Adds response parser tests (based on frontend tests).
Adds time series query tests (based on frontend tests).
Fixes various issues related to response parsing and building search request queries.
Added support for extended stats metrics and geo hash grid aggregations.
2018-05-23 15:09:58 +02:00
77400cef08 elasticsearch: refactor and cleanup
Move time series query logic to specific file.
Remove model parser and move to time series query file, adds parser test.
2018-05-23 14:36:41 +02:00