12 Commits

Author SHA1 Message Date
7a3415148e SQL Expressions: Add cell-limit for input dataframes (#101700)
* expr: Add row limit to SQL expressions

Adds a configurable row limit to SQL expressions to prevent memory issues with large
result sets. The limit is configured via the `sql_expression_row_limit` setting in
the `[expressions]` section of grafana.ini, with a default of 100,000 rows.

The limit is enforced by checking the total number of rows across all input tables
before executing the SQL query. If the total exceeds the limit, the query fails
with an error message indicating the limit was exceeded.

* revert addition of newline

* Switch to table-driven tests

* Remove single-frame test-cases.

We only need to test for the multi frame case. Single frame is a subset
of the multi-frame case

* Add helper function

Simplify the way tests are set up and written

* Support convention, that limit: 0 is no limit

* Set the row-limit in one place only

* Update default limit to 20k rows

As per some discussion here:
https://raintank-corp.slack.com/archives/C071A5XCFST/p1741611647001369?thread_ts=1740047619.804869&cid=C071A5XCFST

* Test row-limit is applied from config

Make sure we protect this from regressions

This is perhaps a brittle test, somewhat coupled to the code here. But
it's good enough to prevent regressions at least.

* Add public documentation for the limit

* Limit total number of cells instead of rows

* Use named-return for totalRows

As @kylebrandt requested during review of #101700

* Leave DF cells as zero values during limits tests

When testing the cell limit we don't interact with the cell values at
all, so we leave them at their zero values both to speed up tests, and
to simplify and clarify that their values aren't used.

* Set SQLCmd limit at object creation - don't mutate

* Test that SQL node receives limit when built

And that it receives it from the Grafana config

* Improve TODO message for new Expression Parser

* Fix failing test by always creating config on the Service
2025-03-11 17:14:33 +00:00
ea71201ddc ServerSideExpressions: Disable SQL Expressions to prevent RCE and LFI vulnerability (#94942)
* disable sql expressions

remove duckdb ref

* Run `make update-workspace`

---------

Co-authored-by: Scott Lepper <scott.lepper@gmail.com>
2024-10-18 11:28:26 +01:00
cbe1e7d63f expr: add explicit strict-mode const (#94838)
expr: add explicit strict-mode value
2024-10-18 10:46:27 +02:00
2cea7af2f5 datasources: querier: handle reducer-settings-mode correctly (#94595)
* datasources: querier: handle reducer-settings-mode correctly

* Try to fix linting error

"unnecessary leading newline (whitespace)"

---------

Co-authored-by: Sam Jewell <sam.jewell@grafana.com>
2024-10-17 08:38:14 +02:00
2403665998 Chore: Remove legacydata package (#88790)
* Chore: Remove legacydata package

* remove unused dependency

* sdk v0.234.0
2024-06-05 22:31:06 +02:00
42b0f802de QueryTypes: Add feature toggle to show query types in datasource apiservers (#88213)
* initial attempt

* show query types

* show query types

* with formatting

* with formatting

* more cleanup

* add feature toggle

* fix build
2024-05-23 18:46:28 +02:00
4cda34ff7d Expressions: Create json schema for query types (#84032) 2024-03-26 06:58:56 +02:00
d82f3be6f7 QueryService: Use types from sdk (#84029) 2024-03-08 18:12:59 +02:00
3036b50df3 Expressions: expose ConvertDataFramesToResults (#83805) 2024-03-04 18:22:56 +02:00
5f6bf93dd5 Expressions: Use enumerations rather than strings (#83741) 2024-03-01 19:38:32 +02:00
70009201d4 Expressions: Sql expressions with Duckdb (#81666)
duckdb temp storage of dataframes using parquet and querying from sql expressions
---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2024-02-27 16:16:00 -05:00
f23f50f58d Expressions: Add model struct for the query types (not map[string]any) (#82745) 2024-02-17 02:59:11 +02:00