30 Commits

Author SHA1 Message Date
38fc22538a Loki: Don't split queries if they use $__range variables (#69852)
* Loki: Don't split queries if they use  variables

* Update public/app/plugins/datasource/loki/querySplitting.ts

Co-authored-by: Matias Chomicki <matyax@gmail.com>

---------

Co-authored-by: Matias Chomicki <matyax@gmail.com>
2023-06-12 11:22:04 +02:00
ee0ee70aa1 Loki: Exclude queries using DISTINCT from query splitting (#69377)
* Query utils: add function to identify distinct queries

* Chore: return false when the expected node is found

* Query splitting: exclude distinct queries from splitting

* Group queries excluded from splitting
2023-06-02 12:50:23 +02:00
67ca91ece3 Loki: Fix incorrect evaluation of real and extracted labels in context (#67112)
* Loki: Fix incorrect evaluation of real/extracted labels in context

* Add tests

* Improve caching and add more tests

* Use mockResolvedValue

* Flip logic for getInitContextFiltersFromLabels

* Update to logic to use fetchSeriesLabels for queries with more than 1 parser
2023-04-24 17:36:30 +02:00
a31104b107 Loki: Refactor logic for using context filters (#66382)
* Loki: Change logic for using context filters

* Dont add parser and parsed labels if multiple

* Update public/app/plugins/datasource/loki/LogContextProvider.ts

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>

* Update public/app/plugins/datasource/loki/LogContextProvider.ts

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>

* Update

* Rename variable

---------

Co-authored-by: Sven Grossmann <sven.grossmann@grafana.com>
2023-04-14 13:44:02 +02:00
663ed7ba82 Loki Query Splitting: Rename from "chunk" to "splitting" (#65630)
* Update chunking mentions in the code and ui

* Rename files and exported functions

* Rename configuration attribute

* Rename grouped querying function name

* Update more function names

* Update unit test

* Update unit tests

* More renames

* Rename time splitting functions
2023-04-03 12:30:08 +00:00
6093e45178 Loki Query Chunking: Refactor naming conventions and reorganize code (#65056)
* Rename splitting files to chunking

* Rename time chunking functions

* Move response functions to response utils

* Remove some blank spaces

* Add an extra test case for frame refIds and names
2023-03-21 15:54:18 +00:00
5d8ec2756e loki: query chunking: better error handling (#64811)
* loki: query chunking: better error handling

* better comment text

Co-authored-by: Matias Chomicki <matyax@gmail.com>

* better comment text

Co-authored-by: Matias Chomicki <matyax@gmail.com>

* changed merge approach

* simplified code

* fixed test

* removed forgotten code

---------

Co-authored-by: Matias Chomicki <matyax@gmail.com>
2023-03-16 08:54:32 +01:00
ede3e9e5c4 Range Splitting: Process instant queries as an independent query group (#64049)
* Query splitting: enable instant queries

* Range splitting: send instant queries as another request group

* Range splitting: increase grouped splitted requests stability

We were defaulting to the `0` index as the first group for the next request batch, but there was no guarantee that the group `0` had a `.partition` entry for `requestN-1`. Now we find the first defined and use that index as the next starting group.

* Range splitting: update unit test
2023-03-07 07:44:13 -05:00
69d3ae8ec1 Range Splitting: ignore empty queries from splitting and fix result resetting (#64053)
Range splitting: ignore empty queries from splitting
2023-03-07 13:09:55 +01:00
192308aac7 loki: query splitting: better stats (#64105) 2023-03-03 04:58:28 -05:00
fdc4973b77 loki: query chunking: consider refId when merging frames (#64103) 2023-03-03 10:06:25 +01:00
40ac0fa14b Query Splitting: Add support for multiple queries (#63663)
* Range splitting: group metric and logs queries

* Range splitting: intercalate logs and metric queries when > 1 queries

* Range splitting: fix possibly undefined error and remove console log

* Range splitting: update test imports

* Range splitting: add unit tests for multiple queries

* Query splitting: use lodash partition function

* Chore: rename variable

* Chore: attempt to improve readability
2023-03-02 10:32:45 +01:00
d56896cf72 Range splitting: Omit instant queries from splitting (#63705)
Range splitting: omit instant queries from splitting
2023-02-27 11:09:46 +01:00
b069fd81b2 loki: query splitting: more robust check (#63542) 2023-02-22 09:16:05 +01:00
8f7b980c95 Loki: query splitting: handle stats (#63436)
* loki: query splitting: handle stats

* update tests
2023-02-20 13:55:56 +00:00
7a8f1ab566 loki: query splitting: stricter types (#63387) 2023-02-17 19:43:08 +01:00
0240f4eb45 Loki Range Splitting: Create a new instance for the initial frame (#63361)
* Range splitting: create a new instance of the initial frame

* Chore: rename variable
2023-02-16 12:55:31 +01:00
84fdb7f908 Loki Range Splitting: Calculate dynamic maxLines per target based on the current response state (#63248)
* Range splitting: update maxLines for logs queries

* Range splitting: add unit tests for dynamic limit requesting
2023-02-13 07:59:20 +01:00
8cd3d1bfda Loki Range splitting: Ignore hidden queries (#63250)
Range splitting: ignore hidden queries
2023-02-10 18:19:17 +01:00
3f144924d1 Loki: Do not chunk when refid contains do-not-chunk (#63276)
add ref-id to not chunk requests
2023-02-10 18:00:49 +01:00
0bd326d846 LogsVolume: Make log volume work with chunking (#63181)
* aggregate logs volume on streaming

* enable log volume queries to be split

* add streaming indicator

* added tests

* also chunk logs queries

* change extraInfo back to undefined
2023-02-10 14:43:41 +01:00
27d70819cc loki: query splitting: split logs queries (#63091)
* loki: calculate logs-chunk-boundaries

* renamed function
2023-02-10 12:01:33 +01:00
94241f6676 Loki Query Splitting: Split queries into sub-queries with smaller time interval (#62767)
* Range splitting: range splitting function

* Range splitting: experiment with 1 hour splits

* Range splitting: reorganize code

* Range splitting: improve code readability and meaning

* Range splitting: add partition limit to prevent infinite loops

* Range splitting: add error handling

* Range splitting: disable for logs queries

* Range splitting: support any arbitrary time splitting + respect original from/to in the partition

* Chore: remove console logs

* Chore: delete unused import

* Range splitting: actually send requests in sequence

* Range splitting: do not split when > 1 query

* Range splitting: combine frames

* Chore: rename function

* split in reverse

* polished reversing

* keep reference to the right frame in the result

* Range splitting: change request state to Streaming

* Range splitting: fix moving only 1 unit of time instead of the provided one

* Chore: change default parameter to timeShift = 1

* Range splitting: do not split for range queqries

* Range splitting: add initial support for log queries

* Range splitting: do not use MutableDataFrame

It has bad performance and it's not required

* Chore: remove unused export

* Query Splitting: move to module

* loki: split: fix off-by-one error (#62966)

loki: split: fix off-by-one loop

* Range splitting: disable for logs volume queries

* Range splitting: combine any number of fields, not just hardcoded 2

* Range splitting: optimize frame-combining function

* Range splitting: further optimize

* Range splitting: combine frame length

* Range splitting: combine stats

* Range splitting: combine stats without assuming the same order

* Query splitting: catch and raise errors

* Range splitting: create feature flag

* Range splitting: implement feature flag

* Range splitting: add unit test for datasource query

* Range splitting: add basic test for runPartitionedQuery

* Range splitting: add unit test for resultLimitReached

* Range splitting: test frame merging

* Chore: fix unit test

---------

Co-authored-by: Sven Grossmann <svennergr@gmail.com>
Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>
2023-02-09 12:27:02 -05:00
b2c8126e6e Loki: Show query size approximation (#62109)
* feat: make api request to /loki/api/v1/index/stats

* fix: add /index/stats to callResource valid urls

* feat: make call to getQueryStats when the query changes

* feat: render user tooltip displaying the estimated value for processed data

* fix: add new props to component tests

* test: add tests for query size estimation

* fix: disable error message on request failure

* refactor: add suggestions from code review

* refactor: only pass required query string
2023-01-27 17:32:53 +00:00
5fe23b4964 Loki: Track obfuscated query (#61325)
* obfuscate query

* also conver numbers and comments

* query obfuscation: simplify obfuscation and keep some identifiers

* Query obfuscation: replace placeholder string with node name

Co-authored-by: Matias Chomicki <matyax@gmail.com>
2023-01-13 16:16:47 +00:00
bd9cce2866 Loki: Add hints for query filters (#60293)
* add hint for label filter

* add hint for line filter

* add feature tracking for hints click

* fix failing tests

* add tests for new query builder hints

* update hint labels

* fix: hint continues to render after adding operation

* add missing title property to hints

* make use of isQueryWithParser and remove isQueryWithLogFmt

* refactor isQueryWithLabelFilter

* always render label operation, even if incomplete

* suggestion

* fix oversight in feature tracking name on hint click

* update query hint label

* improvements

* add missing test for ADD_NO_PIPELINE_ERROR
2023-01-05 13:56:31 +00:00
4be99c56f6 Loki Query Editor: Autocompletion and suggestions improvements (unwrap, parser, extracted labels) (#59103)
* Chore: refactor test to improve internal categorization of scenarios

* feat(loki-monaco-unwrap): add unwrap situation support

* Chore: remove redundant path from aggregation situation

* feat(loki-monaco-unwrap): add unwrap suggestions

* feat(loki-monaco-autocomplete): rename IN_DURATION and add missing tests

* feat(loki-monaco-autocomplete): detect parser and line filter

* feat(loki-monaco-autocomplete): optionally suggest line filters and parsers

* feat(loki-monaco-autocomplete): improve suggestions and update completions tests

* feat(loki-monaco-autocomplete): allow for multiple line filters suggestions

* Chore: update situations test

* Chore: add test case for AFTER_UNWRAP completion

* feat(loki-monaco-autocomplete): use logs query instead of labels for data sample

* Chore: improve getParser function name and add tests

* Chore: update test mock data

* Update public/app/plugins/datasource/loki/components/monaco-query-field/monaco-completion-provider/situation.test.ts

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>

* feat(loki-monaco-autocomplete): improve after unwrap detection

* feat(loki-monaco-autocomplete): remove leftover parser detection

* Chore: completely remove parser suggestion exclusion implementation

It was correct to have more than one parser, so we don't need to exclude parsers if one is present.

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>
2022-12-14 11:37:08 -05:00
d0eeff2fa0 Loki: Add tracking of executed queries (#59887)
* add query tracking

* add app

* add comment

* use `reportInteraction` not `console.log`

* add test to `queryUtils`

* organize imports

* add datasource tests

* change `metrics` to `metric`

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>

* change `parseToArray` to `parseToNodeNamesArray`

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>
2022-12-06 15:54:20 -05:00
46f0672215 Loki: Remove hardcoded values for parsed query parts (#54755)
Co-authored-by: Nitesh Singh <nitesh.singh@gitstart.dev>
Co-authored-by: gitstart <gitstart@users.noreply.github.com>
Co-authored-by: Rafael Toledo <87545086+Toledodev@users.noreply.github.com>
Co-authored-by: Murilo Amaral <87545137+MuriloAmarals@users.noreply.github.com>
Co-authored-by: gitstart <gitstart@gitstart.com>
Co-authored-by: Matheus Benini Ferreira <88898100+MatheusBeniniF@users.noreply.github.com>
Co-authored-by: Matheus Benini <matheus_benini@hotmail.com>
Co-authored-by: Rubens Rafael <70234898+RubensRafael@users.noreply.github.com>
Co-authored-by: Júlio Piubello da Silva Cabral <julio.piubello@gitstart.dev>

Co-authored-by: gitstart <gitstart@users.noreply.github.com>
Co-authored-by: Nitesh Singh <nitesh.singh@gitstart.dev>
Co-authored-by: Rafael Toledo <87545086+Toledodev@users.noreply.github.com>
Co-authored-by: Murilo Amaral <87545137+MuriloAmarals@users.noreply.github.com>
Co-authored-by: gitstart <gitstart@gitstart.com>
Co-authored-by: Matheus Benini Ferreira <88898100+MatheusBeniniF@users.noreply.github.com>
Co-authored-by: Matheus Benini <matheus_benini@hotmail.com>
Co-authored-by: Rubens Rafael <70234898+RubensRafael@users.noreply.github.com>
Co-authored-by: Júlio Piubello da Silva Cabral <julio.piubello@gitstart.dev>
2022-09-16 15:35:20 +02:00
f26babf484 Loki: Rename public/app/plugins/datasource/loki/query_utils.ts file to follow convention (#55180) 2022-09-16 10:16:54 +02:00