691 Commits

Author SHA1 Message Date
34f1be796d unified-storage: expose ring replication factor config (#106345)
* config ring replication factor

* change default

* rename

* fix test

* fix
2025-07-22 09:32:22 +02:00
974a2c47f9 feat(unified-storage): add qos support for the resource server (#105939) 2025-07-01 09:22:55 +00:00
7aad041f70 unified-storage: Add integration tests for distributor (#105771)
* Add integration tests for distributor
2025-06-30 09:31:58 -04:00
0982cfd9a0 Unified Storage/Search: Add max count config for indexing (#107255)
* Add max count config for indexing
* Build empty index when max count is exceeded
* Address linting
* Refactor buildIndexes
* Add test for max count threshold
* Update test doc comments
* Refactor TestBuildIndexes_MaxCountThreshold to not use mock framework
* Rename mocks used in TestBuildIndexes_MaxCountThreshold

* Refactor mockResourceIndex

* Test setting of indexing threshold configs

* Tweak comments, log

* Fix logging in buildEmptyIndex

* Export and reuse TestDocumentBuilderSupplier

* Reuse MockResourceIndex
2025-06-27 14:00:39 +02:00
f6e330b3d7 Alerting: Allow disabling recording rules write for a data source in the UI (#106664)
Adds a new "Allow as recording rules target" toggle to Prometheus datasource configuration that controls whether the datasource can be selected as a target for writing recording rules.
---------

Co-authored-by: ismail simsek <ismailsimsek09@gmail.com>
Co-authored-by: Konrad Lalik <konradlalik@gmail.com>
2025-06-17 22:55:45 +02:00
b48337a7c8 feat: introduce ttl cache for bleve indices (#106842) 2025-06-17 16:28:51 +02:00
1500fa5772 Extend OpenFeature service (#106707) 2025-06-13 17:30:53 +02:00
5f21f320f7 fix(unified-storage): use continue token containing both formats for dualwriter (#106525) 2025-06-13 15:59:46 +02:00
5135d5c87d Unified storage: Reconstruct index in the background every 24h (#106422) 2025-06-12 14:34:48 -05:00
e0d27dc0d7 Dashboard: Add configurable quick ranges for the time picker (#102254)
* Dashboard: Add configurable quick ranges for the time picker

* fix test and linter errors

* update from array to TimeOption

* Switching to grafana-scenes (Part 1 - remove grafana-ui changes

* Update SceneTimePicker initialization

* betterer

* remove hallucinated argument

* Revert "Bump scenes and fix types (#105167)"

This reverts commit c6428dfc7463250f786a9089c46a2fac910333cb.

* make gen-go

* reset files

* Shorten documentation to increase maintainability

* Update _index.md

* the

---------

Co-authored-by: joshhunt <josh.hunt@grafana.com>
Co-authored-by: Jacob Valdez <jacob.valdez@grafana.com>
2025-06-12 17:51:46 +02:00
b38662e985 Plugins: Add preinstall_sync config - allow plugins to be preinstalled synchronously (#105292)
* feat: preinstall_sync config - process and installation logic

* ref: add preinstall_sync list to preinstalled plugins of frontendsettings

* fix: conf blank line for sections

* ref: remove plugins async flag, and rename PreinstallPlugins

* docs: default installed plugin list
2025-05-16 10:45:50 -04:00
5e056c2a3f SQL Expressions: Add sql expression specific timeout and output limit (#104834)
Adds settings for SQL expressions:
 sql_expression_cell_output_limit

Set the maximum number of cells that can be returned from a SQL expression. Default is 100000.

sql_expression_timeout

The duration a SQL expression will run before being cancelled. The default is 10s.
2025-05-13 15:22:20 -04:00
43748e43bb Plugins: Add plugins auto update feature (#104112) 2025-05-09 15:58:04 +02:00
dde613bbe8 Settings: Exclude zanzana client token value from output (#105018)
* Settings: Exclude zanzana client token value from output

* add test

* fix tests
2025-05-06 18:54:48 +02:00
c90e2e8e5e SecretsManager: Add (en/de)cryption packages (#104923)
Merging the code as-is from the feature branch: secret-service/feature-branch

Co-authored-by: PoorlyDefinedBehaviour <brunotj2015@hotmail.com>
Co-authored-by: Dana Axinte <53751979+dana-axinte@users.noreply.github.com>
Co-authored-by: Leandro Deveikis <leandro.deveikis@gmail.com>
Co-authored-by: Mariell Hoversholm <mariell.hoversholm@grafana.com>
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
2025-05-05 15:26:52 +02:00
ecea6a9366 unified-storage: expose memberlist cluster label config (#104643)
* tweak ringKey/ringName for consistency and allow us to configure memberlist cluster label config

* allow setting memberlist cluster label verification disabled field

---------

Co-authored-by: Peter Štibraný <pstibrany@gmail.com>
2025-04-30 09:43:09 -04:00
4adebd6058 unified-storage: setup ring to shard requests (#103783)
* Updates the instrumentation_server service to use mux instead of the builtin router, and have it store the router in the module server: this is so we can register the /ring endpoint to check the status of the ring
* Create a new Ring service that depends on the instrumentation server and declares it as a dependency for the storage server
* Create standalone MemberlistKV service for Ring service to use
* Update the storage server Search and GetStats handler to distribute requests if applicable
2025-04-25 13:08:44 -04:00
ebf3a8d205 fix: make data syncer run configurable on mode (2->3) change (#104401) 2025-04-25 13:43:35 +02:00
d7c554c25e Renderer: Fix regression on callback URL in plugin mode (#103787)
* Renderer: Fix regression on callback URL in plugin mode

* cleanup

* refactor to not mutate cfg object

* apply review feedback
2025-04-11 15:34:10 +02:00
50499fa749 Dashboards: Add a config setting that limits the number of series that will be displayed in a panel. Users can opt in to render all series. (#103405)
* Limit series in panels in dashboards

* fix mangled comment
2025-04-11 13:56:58 +02:00
757be6365a CI: Bump golangci-lint to 2.0.2 (#103572) 2025-04-10 14:42:23 +02:00
f96e4e9ad2 Frontend: Remove Angular (#99760)
* chore(angularsupport): delete feature toggle to disable angular

* feat(angular-support): remove config.angularSupportEnabled

* chore(jest): remove angular from setup file

* chore(angular): delete angular deprecation ui components

* refactor(angular): move migration featureflags into migration notice

* chore(dashboard): remove angular deprecation notices

* chore(annotations): remove angular editor loader

* feat(appwrapper): no more angular app loading

* feat(pluginscatalog): clean up angular plugin warnings and logic

* chore(angular): delete angular app and associated files

* feat(plugins): delete old angular graph plugin

* feat(plugins): delete old angular table panel

* feat(frontend): remove unused appEvent type

* feat(dashboards): clean up angular from panel options and menu

* feat(plugins): remove graph and table-old from built in plugins and delete sdk

* feat(frontend): remove angular related imports in routes and explore graph

* feat(theme): remove angular panel styles from global styles

* chore(i18n): run make i18n-extract

* test(api_plugins_test): refresh snapshot due to deleting old graph and table plugins

* chore(angulardeprecation): delete angular migration notice components and usage

* test(frontend): clean up tests that assert rendering angular deprecation notices

* chore(backend): remove autoMigrateOldPanels feature flag

* chore(config): remove angularSupportEnabled from config preventing loading angular plugins

* chore(graphpanel): remove autoMigrateGraphPanel from feature toggles

* chore(tablepanel): delete autoMigrateTablePanel feature flag

* chore(piechart): delete autoMigratePiechartPanel feature flag

* chore(worldmappanel): remove autoMigrateWorldmapPanel feature toggle

* chore(statpanel): remove autoMigrateStatPanel feature flag

* feat(dashboards): remove automigrate feature flags and always auto migrate angular panels

* test(pluginsintegration): fix failing loader test

* test(frontend): wip: fix failures and skip erroring migration tests

* chore(codeowners): remove deleted angular related files and directories

* test(graphite): remove angular mock from test file

* test(dashboards): skip failing exporter test, remove angularSupportEnabled flags

* test(dashbaord): skip another failing panel menu test

* Tests: fixes pkg/services/pluginsintegration/loader/loader_test.go (#100505)

* Tests: fixes pkg/services/pluginsintegration/plugins_integration_test.go

* Trigger Build

* chore(dashboards): remove angularComponent from getPanelMenu, update test

* feat(dashboards): remove all usage of AngularComponent and getAngularLoader

* chore(betterer): refresh results file

* feat(plugins): remove PluginAngularBadge component and usage

* feat(datasource_srv): remove usage of getLegacyAngularInjector

* feat(queryeditor): delete AngularQueryComponentScope type

* Chore: removes Angular from plugin_loader

* Chore: remove angular from getPlugin

* Chore: fix i18n

* Trigger Build

* Chore: remove more Angular from importPanelPlugin

* Chore: remove search options warning

* Chore: remove and deprecate Angular related

* chore(angular): remove angular dependencies from core and runtime

* chore(runtime): delete angular injector

* chore(data): delete angular scope from event bus

* chore(plugin-catalog): remove code pushing app plugins angular config page

* chore(yarn): refresh lock file

* chore(frontend): remove ng-loader from webpack configs, remove systemjs cjs plugin

* chore(navigation): remove tether-drop cleanup from GrafanaRouter, delete dependency

* chore(runtime): delete AngularLoader

* chore(betterer): refresh results file

* chore(betterer): fix out of sync results file

* feat(query): fix type and import errors in QueryEditorRow

* test(dashboards): delete skipped angular related tests

* Tests: add back tests and fix betterer

* Tests: fix broken test

* Trigger build

* chore(i18n): remove angular deprecation related strings

* test: clean up connections and plugins catalog tests

* chore(betterer): update results file

---------

Co-authored-by: Hugo Häggmark <hugo.haggmark@gmail.com>
2025-04-04 11:31:35 +02:00
77fa2271be AppPlatform: Introduce experimental Github integration for dashboard configuration management (#96329)
* [Provisioning] Pay back some technical debt (#100720)

* Handle pagination in github client

* Add some unit test coverage

* Remove unknown repository

* Remove unknown leftover

* Revert "Add some unit test coverage"

This reverts commit 420c9674d226b65bd5ec8f1f1880517f119d7df0.

* Revert "Revert "Add some unit test coverage""

This reverts commit f7eca419573c889056f7a0ffb2c7bda7f0fa4c9e.

* Revert unit tests in github package

* Remove S3 case as it's now deprecated

* [Provisioning] Consolidate job status report in JobProgressRecorder (#100718)

* Log also successful operation

* Consolidate stop logic under TooManyErrors

* Use error for TooManyErrors

* Pass the progress recorder

* Define JobProgressRecorder interface

* Do not expect workers to return status

* Remove scenarios due to pointers

* Use recorder to manage the entire state

* Provisioning: Support rotating secrets (#100705)

* Provisioning: Refactor webhook to another interface (#100733)

* POC/Provisioning: Remove S3 references (#100734)

* Remove unused script

* Remove s3 references

* Provisioning: Keep the existing k8s name if it is specified in metadata (#100672)

* keep name

* keep name

* Revert "keep name"

This reverts commit 29f87bcaeb4c324dc9e5f5231db066900f35b21a.

* Commit stale go.mod

* Keep name also for sync deletions

---------

Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>

* Update preview banner copy

* Query Library: Move backend to enterprise (#100371)

* Fix wire

* Fix jobs table re-renders

* Provisioning: Refactor history to its own interface (#100735)

* Provisioning: Refactor history to its own interface

* refactor: use VersionedRepository

* Update API

* Provisioning: Test the GitHub client directly (#100808)

* Provisioning: Test the GitHub client directly

Instead of mocking the abstracted client, test it as well by mocking the underlying GitHub client. This also lets us
remove the mock for the abstracted client.

* refactor: move out helpers

* chore: set dependency owner

* Provisioning: Better clone/push error support (#100854)

* Provisioning: Replace searcher with one that knows about modes (#100857)

* Provisioning: Start in "mode5" when nothing exists in legacy (#100862)

* [Provisioning] Fix duplicate sync jobs triggered in controller (#100870)

* Improve logging on reasons why the controller triggered

* Fix messaging for sync job

* fix lint

* Provisioning: Move legacy export/import into a single migrate job (#100865)

* [Provisioning] Miscellanenous bug fixes and improvements (#100976)

* Error if found duplicate ID

* Fix issue with manual test button

* Fix issue with health errors not going away

* Display status in sync overview

* Use patch operations instead

* Trigger sync job after status update

* Convert Export Tab into modal

* Remove unused FieldSet import

* Only last 8 jobs

* Remove Links card

* Use button for Github Source Code

* Add actions to resources page

* Add resource column to Repository Resources

* Display Job Spec in RecentJobs

* Display dates in history page

* Display Avatar if available

* Improve styling of the avatar

* Update betterer

* Remove duplicate history header in history

* Commit betterer

* Address code styling issues

* update flags

* github v69

* v69

* POC/Provisioning: Add wizard (#100596)

* Chore: make update-workspace

* Chore: Fix lints (#101039)

* Provisioning: Workflows as write access (#101031)

* workflow as write access

* workflow as write access

* workflow as write access

* Update pkg/registry/apis/provisioning/repository/test.go

Co-authored-by: Mariell Hoversholm <mariell.hoversholm@grafana.com>

* POC/Provisioning: Add wizard (#100596)

* update refs

* update refs

* lint fix

* lint fix

* lint fix

* default everythign to read only

* reuse form components

* remove main

---------

Co-authored-by: Mariell Hoversholm <mariell.hoversholm@grafana.com>
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* merge main

* Fix workflow types

* Betterer

* [Provisioning] Fix webhook and finalizer issues (#101052)

Fix webhook and finalizer issue

The maximum number of webhooks per repository is 20

* [Provisioning] Fix issue with last ref (#101056)

* Fix issue with last ref

* Update frontend code

* Fix the local tmp test

* Use lastRef

* POC/Provisioning: Simplify connect step (#101064)

* Fix sending workflows

* Use write for local

* Move connect action to the next step

* Remove wizard props

* Typo

* Redirect to wizard

* Show repo link after successful export

* Provisioning: Avoid starting sync jobs when using legacy storage (#101114)

* avoid starting sync jobs on legacy

* newlines

* Provisioning: Onboarding landing page (#101112)

* add landing page before wizard

* Update onboarding page

* Update URL

* Remove unused

* Add deleteAll button

* Improved text

* betterer

---------

Co-authored-by: Clarity-89 <homes89@ukr.net>

* Provisioning: use the sync job to finish the migrate job (#101107)

* Provisioning: Show progress more often (#101128)

* show progress bar earlier

* show progress bar earlier

* update wording to be less specific

* POC/Provisioning: Enable sync (#101131)

* update preview banner

* actualy remove and don't crash without provisioning flag

* Update db banner

* Provisioning: Export oldest items first (#101189)

* Provisioning: better branch handling (#101188)

* add missing file

* Provisioning: Fix tests (#101197)

* Provisioning: Refactor tests to be multiple functions

* Provisioning: Fix tests

* fix: make github-example sync

* fix misspell

* Provisioning: avoid migration wizard if things are already in unified storage (#101204)

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* more lint

* POC/Provisioning: Handle connect step errors (#101192)

* Provisioning: Unify status handling in repository and migrate steps

* Refactor: Move WorkflowsField outside RepositoryStep and use proper type

* Refactor: Improve repository verification error handling and UI

* Refactor: Simplify repository verification error handling

* Refactor: Simplify RepositoryStep component structure

* Refactor: Improve error handling in RepositoryStep

* Refactor: Remove redundant repository creation logic from ProvisioningWizard

* Refactor: Simplify RequestErrorAlert component

* show github error

* now will verify

* test .git

* recover from bad config

* Update error handling

* Remove unused prop

* merge upstream

* Show migration summary

* Update text

* Improve text

* Betterer

* [Provisioning] Review controller changes (#101216)

* Review health check conditions

* Move down the logic to set up the sync status

* Skip if it's only a health check rerun

* Fix health check conditions

* Preserve last ref

* Format code

* Rename to shouldSkipSync

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Roberto Jiménez Sánchez <roberto.jimenez@grafana.com>

* fix promotion step

* In the promotion pipeline, publish should depend on build

* fix promotion pipeline

* [Provisioning] Use smaller methods to process repository events (#101240)

* update codegen

* merge main

* Provisioning: Avoid localhost error loop (#101253)

* Provisioning: Update the recent jobs formatting (#101250)

format history

* [Provisioning] Refactor Pull Request & Lint worker (#101273)

* Refactor the code

* Refactor into separate files

* Consolidate linter flag in one spot

* Use global feature flags

* Commit betterer

* Remove from JSON the intermidiate flag

* Use again spec

* Clean up

* Revert changes in test

* POC/Provisioning: Remove sync confirm modal (#101281)

* [Provisioning] Remove linting from MVP (#101286)

* Remove Linting backend

* Re-generate client

* POC/Provisioning: Unify tags (#101218)

* Unify tags

* add both tags

* add tag types

* Check for the redirect only once

* Add fetch settings with delay hook

* Refetch settings

* Split hooks into separate files

* Cleanup

* Prettier

* Prettier

* Remove lint code

* Betterer

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* Provisioning: Update token instructions (#101280)

* Provisioning: Use blob storage rather than local file system to save images (#101298)

* [Provisioning] Add more explicit setup warnings if webhook integration and image rendering are disabled (#101304)

* Setup warnings if renders or webhooks are not possible

* Improve display

* Use a single Alert

* Make design more compact

* Only display local config is missing critical feature toggles

* Improve styling of required flags

* Add file name to the custom ini itself

* Add copy button

* Add FeatureSection Component

* Commit betterer

* Use an interactive table

* Use a modal for instructions

* Use the same modal for example config

* Improve setup steps

* Improve stepper

* Copy code ctrl + c

* Make it more compact

* Select feature to enable

* Improve the height of alerts

* Separate components

* Better warnings

* Improve the page

* Improve the cards

* Improve cards even more

* Improve cards

* Improve cards

* Optional copy in code block

* Add side bar with steps

* Improve styling

* Style modal

* Clean up Code

* Remove index file

* Simplify Step Component

* Commit betterer

* Simplify components

* Use CodeEditor and Clipboard components

* Do not show scrollback on minimum size

* Fix positioning of footer

* Separate Component for Feature

* Use different styling

* Commit betterer

* Use more Grafana components in the FeatureCard

* Separate sidebar into own components

* Simplify sidebar code

* Commit betterer

* Remove connector

* Simplify styling further

* Use cards

* Improve code

* Use more grafana component in InstructionsModal

* Further simplify

* Simplify the code

* Simplify style

* Clean up

* Simplify the Wizard

* Use little icons

* Improve feature cards

* Improve cards

* Commit betterer

* Add description to feature setup

* Improve instructions for snapshot preview

* Move all files into Setup folder

* Commit betterer

* Clean up the warnings code

* Improve coding

* Move sidebar item to separate fiel

* Rename components

* Fix issues

* Use stack instead

* Improve style

* Don't show setup button if configured already

* Simplify again CSS

* Use secondary actions

* Style a bit more

* Improve wording

* Update warning

* Refer to docs in Image Renderer

* More clean up

* Revert changes in generated client

* Fix typos and imports

* Fix lint errors

* Provisioning: better error support (#101490)

* update openapi snapshot

* fix build

* Provisioning: Only show setup page when feature toggles are missing (#101502)

* form fixing

* form fixing

* always send UID

* Same onboarding page regarless of migration (#101557)

* backend building... frontend still broken

* rename sync with main

* Provisioning: Update dashboard badge (#101599)

* Rename to push / pull everything user-facing (#101577)

* Rename to push / pull all everything user-facing

* Use automatic pulling wording

* Provisioning: Migrate when using unified storage (#101572)

* migrate when not unified

* Update pkg/registry/apis/provisioning/register.go

Co-authored-by: Roberto Jiménez Sánchez <roberto.jimenez@grafana.com>

* variables

* merge main

---------

Co-authored-by: Roberto Jiménez Sánchez <roberto.jimenez@grafana.com>

* Add tabs and features tab to listing page (#101570)

* List features in onboarding page (#101558)

* merge main

* POC/Provisioning: Check if the instance is provisioned (#101601)

* Check if the instance is provisioned

* Fix lints

* Fix getting config for new dashboard

* Fixes after merge

* More fixes

* Show success message

* Fix default value

* Add test

* Fix lints

* Provisioning: Include URLs in ResourceWrapper response (#101511)

* Convert Migrate wizard into a Connection Wizard (#101575)

* Convert Migrate wizard into a Connection Wizard

* Remove duplicate empty state

* Allow users to select target in the first step

* Remove file created by merge

* Select target based on existing connections

* Default option for targets and explainatory alert

* Do not display connect button if single connection

* Display target as tag in repository card

* Add Pull Step

* Fix linting

* User decides if migrate or connect

* Improve style based based on review

* Provisioning: Return upsert resource when writing (#101574)

* [Provisioning] Getting Started Page and Tab (#101701)

* merge main

* fix go.mod

* Provisioning: Redirect to the new URL after save (#101757)

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* POC/Provisioning: create folder (#101619)

* Add NewProvisionedFolderForm

* Cleanup

* Add folder api

* Register API

* Do not show provisioned badge for instances

* Enable saving

* Show saved alert

* Fixes

* Fix deps

* Cleanup

* Add test

* Add test

* Updates

* Fix test

* Fix import

* [Provisioning] Display instance connection directly in home (#101720)

* Show tabs directly in home page for instance sync

* Display actions also in the home page

* Add delete button to actions

* Fix issue with files tab

* Display tabs also in instant sync

* Generate connection name for instance sync

* Fix issues when no repositories

* Set active tab

* Remove leftover

* Rename component for listing folder repository

* Fix linting issues

* Fix merge error

* Rename to HomePage

* Split folder list into separate component

* Create own component for repository card

* Improve RepositoryCard readability

* Improve RepositoryCard styling

* Make view primary button

* Fix syntax

* Fix generateName

* merge main

* Provisioning: Move folder management into its own helper (#101864)

* folder

* update folder links

* Fix test

* update

* cleanup

---------

Co-authored-by: Clarity-89 <homes89@ukr.net>

* fixed folder issue

* POC/Provisioning: Create folder from root (#101921)

* Enable creating folder at the root

* Fix test

* Add interceptor

* Provisioning: Expose stats (#101927)

* [Provisioning] Unified onboarding wizard (#101952)

* Spike the solution

* More work

* Add more situations

* Attempt to display count of dashboards and folders

* Attempt with file count

* Do not display options if not possible

* Improve styling resources

* Use another API

* Fix issue with selection

* Style a bit

* Fix more issues

* Make the sync step work

* Improve links

* Use LinkButton

* Start pull automatically

* Start migration automatically

* Fix issue with options

* Fix issues

* Fix loading error

* Improve more things

* Improve styling

* Improve messaging

* Set the autofocus

* Fix some issues

* Fix issue with disabled options

* Only resources

* Finish settings depending on configuration

* Move title to wizard

* Fix title

* Improve styling

* Badge

* Explain on hover

* Improve styling

* Disabled at the bottom

* History & identifiers

* Improve wording

* Add padding left and right disable options

* Delete repository

* Improve buttons

* Give index time to catch up

* Improve buttons

* Handle steps with only forms

* Fix issue with initial migrate or pull

* Commit betterer

* Error messages

* Use memo

* Revampt that a bit

* Attempt to simplify the state and components

* Improve the component for Migrate

* Commit betterer

* Fix issue in next button

* Clean up more

* Start for boostrap step

* Fix issue with running status

* Fix issue with loading bootstrapping

* Improve loading

* Improve more the loading

* Fix issue with loading

* Empty tree

* Handle error

* Fix issue with looping

* Remove commented out lines

* Add comment

* Remove accidental file

* Fix imports

* Improve MigrateStep and PullStep

* Use hook for step status

* JobStep component

* Refactor data fetching

* Validate with Github

* Fix issue with failed error

* Fix next on success

* Address small comments

* Separate file for WizardContent

* Fix linting

* Use step approach also for bootstrap

* Make the logic for moving between steps clearer

* Fix navigation issue

* Clean up some logic

* Use useAsync for JobStep steps

* Revert "Use useAsync for JobStep steps"

This reverts commit 242a275cc94a2f4d61eb338d7e56e7186357a3ae.

* Provisioning: use service to get counts (#101972)

counts

* must migrate when using legacy storage

* Revert "Revert "Use useAsync for JobStep steps""

This reverts commit a420d0ac364a2b97036f585c18b9f3781864f37a.

* Fix async conditions

* Organize imports

* Separate component for BootstrapOptionCard

* BootstrapOptionsList

* Remove duplicate definitions

---------

Co-authored-by: Clarity-89 <homes89@ukr.net>
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* Merge

* Fix utils

* POC/Provisioning: Fix folder path for dashboard (#101997)

* Provisioning: Fix folder path for dashboards

* Fix isNew

* Update test

* Fix any error

* Betterer

* [Provisioning] Improve progress recording and updates (#102035)

* do not validate on delete

* Provisioning: Implement authorizer for remaining resources (#101945)

* feat: implement authorizer for remaining resources

* fix: don't allow viewers to write files

* security: harden blob id fetching

* add integration test for admin vs viewer

* feat: only Get is a valid verb for reads in our subresources

Co-Authored-By: Ryan McKinley <ryantxu@gmail.com>

* feat: allow render for all requests

* refactor: use guards

Not changing code that goes `if a { } else if b { } else { }` as the semantic meaning of the different branches is
easier to parse.

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* POC/Provisioning: Improve wizard setup (#102066)

* wizard actions

* workign better

* remove more memo

* show polling interval

* cleanup

* finalizers

* Update public/app/features/provisioning/Wizard/BootstrapStep.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* Update public/app/features/provisioning/Wizard/BootstrapStep.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* Update public/app/features/provisioning/Wizard/BootstrapStep.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* Update public/app/features/provisioning/Wizard/BootstrapStep.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* Update public/app/features/provisioning/Wizard/BootstrapStep.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* Update public/app/features/provisioning/Wizard/WizardContent.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* Update public/app/features/provisioning/Wizard/WizardContent.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* Update public/app/features/provisioning/Wizard/BootstrapStep.tsx

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>

* updates from alex

* updates from alex

* Simplify actions

* Extract props

* history supported form legacy only (for now)

* More refactor

* change order

* Fix cleanup finalizer

* show kinds

* fix lint

---------

Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
Co-authored-by: Clarity-89 <homes89@ukr.net>
Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>

* Fix resource condition (#102086)

* Fix job summary stats issue (#102084)

* merge main

* Repository link should point to configured branch (#102092)

* Add getRepoHref

* Fix random string generation

* Redirect to Home on repository deletion (#102096)

* Fix extra commas in pull request comment (#102108)

* Fix image rendering endpoint (#102107)

* POC/Provisioning: Support migrate... when starting with unified storage (#102097)

* use same clone

* now using upsert

* Fix lint

---------

Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>

* Clean up unprovisioned resources after unified storage migration (#102126)

* Clean up unprovisioned resources after migrate

* Clean up unprovisioned resources after migrate

* Update pkg/registry/apis/provisioning/jobs/migrate/resources.go

* Reset summary between export and pull (#102101)

* Reset summary between export and pull

* Add reset results to unified storage migration

* Provisioning: always dirty (#102151)

* fix test version

* log the watch line

* POC/Provisioning: Disable repository list watch (#102169)

* Disable watch for repo list endpoint

* Add comment

* Remove another watch

* Provisioning: Avoid calling test on every update (#102161)

test less often

* Provisioning: Support prefixes in GitHub repositories (#101969)

* feat: add a Prefix property to GitHub repo spec

* feat: make nested folders work properly

* feat: use subdir for go-git export

* fix: placeholder for prefix should be grafana/

* feat: rename prefix to path

* fix: json name should be path, too

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* chore: regen apis

* fix: copy 'path'

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* Merge

* Fix duplicate type

* Provisioning: Test export functionality (#101336)

* Provisioning: Test export functionality

* fix: use context.Background

* test: add more cases for local repo path resolving

* test: rework test inputs

* test: try to make github test work

* fix: clear global state

* Update api client imports

* Fix import

* Fix test

* Update codegen

* Provisioning: Make it green (#102271)

* chore: remove unused functions

* chore: update betterer results

* chore: update openapi spec

* chore: yarn generate-apis

* fix: specify default false if undefined

* Use AnnoKeyManagerIdentity

* Add manager kind

* POC/Provisioning: Update component structure (#102297)

* Update project structure

* Update imports

* Remove unused components

* Copy fixes

* Typo

* More copy fixes

* Betterer

* Update test

* merge main

* Provisioning: Replace hardcoded clients with discovery client (#101918)

* disco client

* discovery client

* merge main

* merge main

* keep factory

* keep factory

* find preffered version for delete factory

* use same folders request

* merge main

* with integration test

* POC/Provisioning: Compare spec in test rather than raw JSON (#102352)

* compare spec not json

* compare spec not json

* [Provisioning] Add in-code TODOs in API Server area (#102360)

* Add TODOs for files endpoint

* Add TODO history endpoint

* Add TODO to move files logic to resource package

* Add TODO to not use private fields directly

* Remove unnecessary checks in list connector

* Add pagination TODO in lister

* Add TODO to rename resources

* Add todo about cloning too early

* Add TODO to propose to merge sync and migrate endpoints

* Add TODOs in register

* Add more TODOs in connectors & routes

* Add TODOs about prefix

* Change it to remove

* Update pkg/registry/apis/provisioning/test.go

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* POC/Provisioning: Switch folders to use managedBy (#102362)

* Folders: Switch to managedBy

* Fix create folder

* Fix tests

* Do not allow changing folder from dashboard settings

* Update imports

* Update provisioned meta

* Do not show provisioned badge for child folders

* Fix folder title

* Update folder actions

* Update new provisioned folder form

* Remove unused code

* Fix condition

* Reset default values on change

* Remove duplicate nav item

* Add managedBy to DashboardQueryResult

* Provisioning: support watch over live (in feature branch) (#102408)

* Provisioning: watch cleanup (#102424)

* fix lint

* Provisioning: Add basic usage stats (#102405)

* [Provisioning] Add limitations to Github Repository (#102451)

* Put limits to Github

* File is too large

* Move constants

* Embed ListOptions again

* Remove TODO

* Provisioning: Pick a better default title (#102516)

better title

* Provisioning: sanitize pull request urls (#102517)

* [Provisioning] Clean up clone after export and migrate (#102467)

* Remove clone directory on clone failure

* Defer remove clones

* Log error if removal fails

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* [Provisioning] Limit path length and depth in APIs (#102472)

* Limit filepath length in files API calls

* Add common utility to deal with paths

* Use the existing function

* Fix import

* Update pkg/registry/apis/provisioning/safepath/limit.go

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* Fix issue after website commit

* Fix linting issue in test

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* Add timeout, JSON check and max body size to endpoints (#102443)

* Add timeout, JSON check and max body size to endpoints

* Use http.MaxBytesReader instead

* Use MaxBytesReader also for reading the entire body

* Add empty line

* Add unit tests

* Fix integration tests

* Update pkg/registry/apis/provisioning/render.go

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* Do not use utils :)

* Fix comment on unmarshalJSON

* 25MB for webhook events

* Remove content type check for files write

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* fix imports

* Provisioning: Remove export option from the UI (#102511)

* [Provisioning] Remove unused checkout method in go-git (#102460)

* [Provisioning] Limit max number of repositories to 10 (#102542)

* Limit to maximum 10 repositories in backend

* Change messaging

* Do not display connect button if more than 10

* Only fetch settings once

* watch repos

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* [Provisioning] Limit size and time to git clone and push in go-git (#102458)

* Limit git clone and push time and size

* Fix linting

* Use transport instead to limit

* Remove not supported

* Add TODO to make timeout configurable

* chore: make update-workspace

* Provisioning: Implement a new job queue (#102446)

* feat: implement a new job queue

Outstanding problems:

  * Status isn't saved.
  * Progress updates don't work (due to status not being saved probably?).

* feat: properly save status

* chore: document label

* chore: assumptions do hold

* fix: support multi-tenant job drivers

* fix: use namespace=*

* fix: set resource back to pointer when updating job progress

If we don't do this, we start rejecting job progress updates as the version falls out of sync.

* feat: make job APIs read-only

* fix: complete job when worker returns

* fix: set namespace on requests from controller

* test: check historic jobs

* chore: regen apis

* feat: start augmenting frontend

* feat: add jobs to authorizer

* feat: use watch from input

* fix: make frontend subscribe to historic jobs

* fix: lint

* chore: yarn prettier:write

* fix: frontend lints

* test: allow for empty state in historicjobs

* test: set content type for export request

* fix: always set job name on insert

* fix: import

* fix: use dashes not colons

* fix: job status should expect a historic job transition

* fix: allow PR jobs from multiple PRs

* feat: same name for sync and migrate jobs

* feat: generate a job name in the store

* refactor: rename to persistentStore

* feat: remove status subresources on jobs

* feat: join jobs into one card

* chore: regen openapi snapshot

---------

Co-authored-by: Roberto Jimenez Sanchez <roberto.jimenez@grafana.com>

* Provisioning: Use a complete storage for jobs (#102605)

* feat: add a complete strategy to apiserver

* feat: use the complete storage strategy for jobs

* test: behaviour changed in main

* [Provisioning] Consolidate file path handling (#102617)

* Add more cases for validation

* Call the method dir

* Clean files endpoint

* Simplify further

* Fix issues with folder

* Add Dir function

* Use walk function in folders

* Move things from ID

* Fix some tests

* Add tree

* Sync worker and changes

* Add more TODOs

* Add normal join

* Remove things in local

* Consolidate single Join

* Call it safe

* Add new IsPathSupported action

* Move the depth to resources

* Add more cases

* Improve trie implementation

* Add tests trie

* Fix trie tests

* Improve trie tests

* Add tests for walk

* Fix linting

* Add unit tests filepath

* Remove TODO

* Remove another TODO

* Unsupported file extension error

* Add documentation for IsPathSupported

* Filepath unit tests

* Use safepath to validate github path

* Remove TODO in wrapper

* Use trailing slash in folder internal object

* Fix changes test

* Include dot

* Add TODO to explore own type for path

* Fix frontend lint

* Fix unit tests

* Fix provisioning integration tests

---------

Co-authored-by: Roberto Jiménez Sánchez <roberto.jimenez@grafana.com>
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Clarity-89 <homes89@ukr.net>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: Kevin Minehart <kmineh0151@gmail.com>
2025-03-25 08:59:03 +01:00
543c0bbccb App platform: Add cleanup job for dashboards when going through /apis (kubectl) (#102506)
* Add dashboard cleanup job

Change log message

Adjust logic to account for new head RV logic

Don't update lastResourceVersion due to pagination

Save improvements

* Address review feedback

* Update docs.

* Remove docs

* Rename config

---------

Co-authored-by: Marco de Abreu <18629099+marcoabreu@users.noreply.github.com>
2025-03-22 23:47:27 +01:00
94020aa9cd Config: Remove setting editors_can_admin (#101607)
* Remove editors can admin

* delete file

* add ac_test back

* Add linting ignore

* more static ignore
2025-03-12 16:25:23 +00:00
00d9916113 ExploreMetrics: Preinstall metrics drilldown app (#100094)
* feat: preinstall metrics drilldown app

* refactor: simplify

* fix: read feature toggles before plugin settings
2025-03-12 09:23:10 -04:00
7a3415148e SQL Expressions: Add cell-limit for input dataframes (#101700)
* expr: Add row limit to SQL expressions

Adds a configurable row limit to SQL expressions to prevent memory issues with large
result sets. The limit is configured via the `sql_expression_row_limit` setting in
the `[expressions]` section of grafana.ini, with a default of 100,000 rows.

The limit is enforced by checking the total number of rows across all input tables
before executing the SQL query. If the total exceeds the limit, the query fails
with an error message indicating the limit was exceeded.

* revert addition of newline

* Switch to table-driven tests

* Remove single-frame test-cases.

We only need to test for the multi frame case. Single frame is a subset
of the multi-frame case

* Add helper function

Simplify the way tests are set up and written

* Support convention, that limit: 0 is no limit

* Set the row-limit in one place only

* Update default limit to 20k rows

As per some discussion here:
https://raintank-corp.slack.com/archives/C071A5XCFST/p1741611647001369?thread_ts=1740047619.804869&cid=C071A5XCFST

* Test row-limit is applied from config

Make sure we protect this from regressions

This is perhaps a brittle test, somewhat coupled to the code here. But
it's good enough to prevent regressions at least.

* Add public documentation for the limit

* Limit total number of cells instead of rows

* Use named-return for totalRows

As @kylebrandt requested during review of #101700

* Leave DF cells as zero values during limits tests

When testing the cell limit we don't interact with the cell values at
all, so we leave them at their zero values both to speed up tests, and
to simplify and clarify that their values aren't used.

* Set SQLCmd limit at object creation - don't mutate

* Test that SQL node receives limit when built

And that it receives it from the Grafana config

* Improve TODO message for new Expression Parser

* Fix failing test by always creating config on the Service
2025-03-11 17:14:33 +00:00
a112ef6467 Authz: Add deprecation notice for settings viewers_can_edit and editors_can_admin (#100947)
* deprecate settings viewers_can_edit editors_can_admin

* add back variables for tests

* delete the files from cached in gti
2025-02-21 11:17:11 +00:00
6200361f36 Auth: Add IP address login attempt validation (#98123)
* Auth: Add IP address login attempt validation

* LoginAttempt struct IpAddress field must be camelCase to match db ip_address column

* add setting DisableIPAddressLoginProtection

* lint

* add DisableIPAddressLoginProtection setting to tests

* add request object to authenticate password test

* nit suggestions & rename tests

* add login attempt on failed password authentication

* dont need to reset login attempts if successful

* don't change error message

* revert go.work.sum

* Update pkg/services/authn/clients/password.go

Co-authored-by: Misi <mgyongyosi@users.noreply.github.com>

---------

Co-authored-by: Misi <mgyongyosi@users.noreply.github.com>
2025-02-05 20:16:36 +02:00
408e3e91a8 Live: make maximum WebSocket message size configurable (#99770)
Co-authored-by: Chris Marchbanks <chris.marchbanks@grafana.com>
2025-01-30 10:24:29 +00:00
336449c169 Explore: Add hide_logs_download and hide button to download logs (#99512)
* Explore: Add `disableLogsDownload` and hide button to download logs

* change copy

* Explore: Change `disableLogsDownload` to `hide_logs_download`

* change casing in frontend

* also hide from inspector

* add test

* lint
2025-01-29 11:53:52 +01:00
056b5a7b08 Dashboards: Monitor dashboard loading performance (#99629)
* WIP benchmark dashboard rendering

* Script

* Benchmark with variable and a panel

* Add one more benchmark

* Explicitely enable profiling

* Playwright tests

* update scenes

* Report measurement to faro when config set

* Let user enable metrics reporting in UI

* Fix logging

* Change how performance metrics is enabled per dashboard, now in config file only

* add benchmark run option

* Fix benchmark runs

* fix description for performance config

* remove console.log

* update codeowners

* add back crashDetection init that was lost in merge

* fix yarn.lock

* restore custom.ini

* fix import

* Make sure we have the echoSrv

* fix config type

* Try to limit changes to e2e runs

* remove benchmark

* Fix lint issue

* fix codeowners

---------

Co-authored-by: Dominik Prokop <dominik.prokop@grafana.com>
Co-authored-by: Sergej-Vlasov <sergej.s.vlasov@gmail.com>
2025-01-28 18:36:10 +01:00
be8396cafa Setup legacy search based on mode (#98908) 2025-01-27 12:32:07 -06:00
865e911e10 Share: Add analytics to invite user flow (#99116) 2025-01-21 11:47:57 -03:00
2187a66f2b Zanzana: Split up settings into client and server sections (#99066)
* Split up zanzana settings into client and server sections

* Update workspace
2025-01-16 13:39:39 +01:00
70fb7b9545 Plugins: Remove bundled plugins feature (#96490)
* remove bundled plugins

* Add echo to command

Co-authored-by: Andres Martinez Gotor <andres.martinez@grafana.com>

* remove folder entirely

* remove folder from CODEOWNERS

---------

Co-authored-by: Andres Martinez Gotor <andres.martinez@grafana.com>
2025-01-14 12:04:01 +00:00
4e398bf2bd Datasources: Add toggle to control default behaviour of 'Manage alerts via Alerts UI' toggle (#98441)
* Datasources: Add toggle to control default behaviour of 'Manage alerts via Alerts UI' toggle

* Update documentation with suggestions

Co-authored-by: Larissa Wandzura <126723338+lwandz13@users.noreply.github.com>
2025-01-10 15:20:09 +01:00
e6e5d23259 Unified Storage: Adds settings for ca cert path and skipping cert verification (#98412)
Adds settings to US for ca cert path and skipping cert verification
2025-01-06 14:00:38 -06:00
4837585cab Unified Storage Search: Add sprinkles (#97820)
* Wire up sprinkles to oss and enterprise. Fetching sprinkles not implemented yet.

* Adds wireset for initializing document builders. Had to init it when creating the service to avoid cyclical imports.

* updates to int64 for stats

* adds config for sprinklesApiServer and gets sprinkles from there when its present

* add comment for later

* adds feature toggle for sprinkles. returns empty results when flag not enabled.

* adds unified storage config setting for sprinkles apiserver page limit

* fixes bug where dashboard uid was not getting set

* when creating dashboard summary, use metadata.name as the dashboard uid

* cleans up wire. use existing oss and enterprise sets to generate doc builders

* remove old wireset

* fix linter - adds missing arg for doc builders

* update dashboard stats in tests

* updates test-data dashboards

* log a warning instead of returning an error if we can't get sprinkles for a namespace

* dont read uid from dashboard json
2024-12-18 11:00:54 -06:00
Jo
40d3b02648 Auth: Separate anonymous settings to its own struct (#97791)
separate anonymous settings to its own struct
2024-12-13 10:46:27 +01:00
1509fadd5b refactor(unified-storage): make datasyncer configurable (#97832) 2024-12-12 21:41:01 +01:00
d9cd5a3a2c Setting: Fix redact of entity db pass (#97568) 2024-12-06 17:31:36 +01:00
70c9c3889f Unified Storage Indexer: Add back metrics (#97310)
* Adds back indexer metrics. Uses config values instead of hardcoded ones.

* cast to int64

* remove unused func

* Index metrics impl doesn't depend on Bleve. Adds a TotalDocs func to SearchBackend interface.

* adds config setting for index_min_count

* rename arg

* rename metric label to namespace instead of slug

* adds default "do nothing" case to satisfy linter

* moves bleve index metrics to search package

* make bleve backend private, dont need to pass in prom reg

* imports

* adds bleve metrics to resource package to avoid circular deps
2024-12-04 15:02:40 -06:00
ef79450065 Settings: Redact token keyword (#97395)
* Settings: Redact token keyword

* Be more specific

* Given section is part of key, be more specific

* Fix test
2024-12-04 16:58:29 +01:00
5e91984f31 Plugins: Use grafana-com sso_api_token (#97096) 2024-12-02 16:04:05 +01:00
e0935246a3 Preinstall: Allow to set a download URL (#96535) 2024-11-29 16:02:33 +01:00
b2626a2d65 LoginAttempt: Add setting to control max number of attempts before user login gets locked (#97091)
* Add setting to adjust number of login attempts before user login gets locked

* Ensure at least one attempt can be made

* Update documentation with new setting

* Update docs/sources/setup-grafana/configure-grafana/_index.md

Co-authored-by: Eric Leijonmarck <eric.leijonmarck@gmail.com>
2024-11-28 14:33:18 +01:00
aa2b4751a0 AuthZ: Launch service within IAM app (#96421) 2024-11-20 11:13:33 +01:00
6abe99efd6 Auth: Passwordless Login Option Using Magic Links (#95436)
* initial passwordless client

* passwordless login page

* Working basic e2e flow

* Add todo comments

* Improve the passwordless login flow

* improved passwordless login, backend for passwordless signup

* add expiration to emails

* update email templates & render username & name fields on signup

* improve email templates

* change login page text while awaiting passwordless code

* fix merge conflicts

* use claims.TypeUser

* add initial passwordless tests

* better error messages

* simplified error name

* remove completed TODOs

* linting & minor test improvements & rename passwordless routes

* more linting fixes

* move code generation to its own func, use locationService to get query params

* fix ampersand in email templates & use passwordless api routes in LoginCtrl

* txt emails more closely match html email copy

* move passwordless auth behind experimental feature toggle

* fix PasswordlessLogin property failing typecheck

* make update-workspace

* user correct placeholder

* Update emails/templates/passwordless_verify_existing_user.txt

Co-authored-by: Dan Cech <dcech@grafana.com>

* Update emails/templates/passwordless_verify_existing_user.mjml

Co-authored-by: Dan Cech <dcech@grafana.com>

* Update emails/templates/passwordless_verify_new_user.txt

Co-authored-by: Dan Cech <dcech@grafana.com>

* Update emails/templates/passwordless_verify_new_user.txt

Co-authored-by: Dan Cech <dcech@grafana.com>

* Update emails/templates/passwordless_verify_new_user.mjml

Co-authored-by: Dan Cech <dcech@grafana.com>

* use &amp; in email templates

* Update emails/templates/passwordless_verify_existing_user.txt

Co-authored-by: Dan Cech <dcech@grafana.com>

* remove IP address validation

* struct for passwordless settings

* revert go.work.sum changes

* mock locationService.getSearch in failing test

---------

Co-authored-by: Mihaly Gyongyosi <mgyongyosi@users.noreply.github.com>
Co-authored-by: Dan Cech <dcech@grafana.com>
2024-11-14 08:50:55 -05:00
Jo
ddf766567b RemoteCache: Refactor remote cache settings (#95672)
* refactor remote cache settings

* fix cache error getting treating as application error

* fix cache error getting treating as application error
2024-11-04 17:35:31 +01:00