* Get ResourceStats before indexing
* Replaced localcache.CacheService to handle expiration faster (localcache.CacheService / gocache.Cache only expires values at specific interval, but we need to close index faster)
* singleflight getOrBuildIndex for the same key
* expire only in-memory indexes
* file-based indexes have new name on each rebuild
* Sanitize file path segments, verify that generated path is within the root dir.
* Add comment and test for cleanOldIndexes.
* Fix CodeQL warnings
* Also validate if path is a file above the safe dir
* Lint
* [REVIEW] reduce possibility of exploits
* Comment
* Remove test scenario
* refactor grafana_index_server_index_size to calculate in a goroutine instead of at scrape time and remove grafana_index_server_indexed_docs metric
* use wire to inject bleve index metrics
* remove sprinkles metrics from bleve index metrics
* log error when trying to calculate file index size and bump interval to 1m instead of 5s
* wires up dashboards page to be able to sort by usage stats (sprinkles)
* dont mutate field
* use better type for field
* adds tests. Had to export some types and put the field type back to object.
* frontend asks for sort field in response if needed
* adds some unit tests for getSortOptions
* use Record instead of object
* prettier
* adds ternaries, another unit test
* add feature flag
* puts search permission filtering behind a feature flag
* fixes issue with doc match id. When the match is for an in-memory index, the internal id is a string (this is what we expected). However, when its a file-based index, the internal id is a binary encoded int64 that point to something internally. So to get the id, we need to use ExternalID() instead of relying on the indexInternalID to be the correct format.
* adds debug log
* update comment
* formatting
* fix bug when parsing results in search handler
* applies permissions filtering to bleve query
* formatting
* wraps in check for access being present, adds some comments
* update go mod
* fix tests
* add dep owner
* fix go mod
* add space after //
* clean up returns
Co-authored-by: Bruno Abrantes <bruno.abrantes@grafana.com>
* fixed formatting
* Uses single checker since index is for single resource. Passes folderId using dvReader to checker func. Adds debug logging.
* handles federation with index permission checkers
* formatting
* move import
---------
Co-authored-by: Bruno Abrantes <bruno.abrantes@grafana.com>
* wip. adding sprinkles fields.
* some refactoring. Works with sprinkles now.
* exclude top level dashboard hit fields from hit "fields"
* adds unit test for DecodeCell helper
* test can search for specific dashboard fields on bleve index
* adds search handler tests for the fields and tests for fields when transforming the search req to a bleve search req
* fix panic when calling fields.Set() with int32
* adds regression test
* remove unneeded method on test mock client
* fix linter issues
* updates dashboard test data for bleve tests
* remove DASHBOARD_LEGACY_ID from bleve_tests
* dont cast twice
* updates test to sort by dashboard_views_last_1_days
* declare excludedFields outside of function
* fixes sorting by dashboard fields - prepends "fields." to any dashboard fields we try to sort by
* uses map for excludedFields
* expects fields to be array-style url param
* change method name
* fixes failing tests - needed to add column type to mocks
wire up the ui to the new search api
Co-authored-by: Scott Lepper <scott.lepper@gmail.com>
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Dan Cech <dcech@grafana.com>
* Wire up sprinkles to oss and enterprise. Fetching sprinkles not implemented yet.
* Adds wireset for initializing document builders. Had to init it when creating the service to avoid cyclical imports.
* updates to int64 for stats
* adds config for sprinklesApiServer and gets sprinkles from there when its present
* add comment for later
* adds feature toggle for sprinkles. returns empty results when flag not enabled.
* adds unified storage config setting for sprinkles apiserver page limit
* fixes bug where dashboard uid was not getting set
* when creating dashboard summary, use metadata.name as the dashboard uid
* cleans up wire. use existing oss and enterprise sets to generate doc builders
* remove old wireset
* fix linter - adds missing arg for doc builders
* update dashboard stats in tests
* updates test-data dashboards
* log a warning instead of returning an error if we can't get sprinkles for a namespace
* dont read uid from dashboard json
* Adds back indexer metrics. Uses config values instead of hardcoded ones.
* cast to int64
* remove unused func
* Index metrics impl doesn't depend on Bleve. Adds a TotalDocs func to SearchBackend interface.
* adds config setting for index_min_count
* rename arg
* rename metric label to namespace instead of slug
* adds default "do nothing" case to satisfy linter
* moves bleve index metrics to search package
* make bleve backend private, dont need to pass in prom reg
* imports
* adds bleve metrics to resource package to avoid circular deps
