53 Commits

Author SHA1 Message Date
c4d3eb1cd0 Remove support for Google Spanner database. (#105846)
* Remove support for Google Spanner database.
2025-05-23 11:35:59 +02:00
694b9dfe50 Chore: Replace xorm.io/xorm imports (#104458)
* replace xorm.io/xorm imports

* replace xorm from other go.mod files

* clean up workspace

* nolint does not make sense anymore as it is not a module

* try if nolint directive helps

* use nolint:all for xorm

* add more nolints

* try to skip xorm in linter config

* exclude xorm differently

* retrigger ci
2025-05-02 17:13:01 +02:00
f7f28757f6 Skip noop migrations, instead of executing SELECT 0. (#103086)
* Skip noop migrations, instead of executing SELECT 0.
2025-03-31 15:19:32 +02:00
0ce67dad74 Spanner: support for lastInsertId (#101770)
* Support for returning inserted ID in Spanner.
* Convert bytes to string when storing them into text column.
2025-03-10 15:59:06 +01:00
9e690bd620 Spanner support enhancements (#101634)
* Adds ability to run integration tests against spanner (by using GRAFANA_TEST_DB=spanner env variable. SPANNER_DB variable then specifies database to use: spannertest, emulator or string like /projects/<project>/instances/<instance>/databases/<db>)
* Adds feature to migration dialects to create database from a snapshot, instead of running individual migrations.
* Adds first version of Spanner snapshot, prepared from "OSS" migrations.
* Uses generated bit-reversed-positive values instead of auto_increment. (As an experiment)
2025-03-06 16:11:20 +01:00
165bca6417 Chore: Add initial/experimental xorm spanner driver (#101398)
* make it build and start

* run some migrations

* add build tags, remove log

* remove unused code

* revert go.mod changes

* move initialisation into dialect file

* update workspace

* update workspace once again

* clean up dependencies

* further cleanup

* Address some review feedback.

* Fix go.sum.

---------

Co-authored-by: Peter Štibraný <pstibrany@gmail.com>
2025-03-03 17:02:10 +01:00
ced7a84f8b chore(metrics): Add metrics & traces to DB migration (#97181)
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
2024-12-05 17:22:19 -05:00
5dfe4cf407 Storage: Watch tests (#85496)
* basic watch tests working

* refactor to read previous event within poller

* add watch test files

* cleanup

* watch tests passing

* debug cleanup

* special handling for canceled context

* cleanup

* fix wire

* fix sqlite_sequence cleanup issue

* move watch tests to integration, wait for provisioned dashboards

* handle context deadline exceeded and eof errors

* add comment about sleep
2024-04-08 11:42:12 -04:00
1f1461734c Storage: Add support for sortBy selector (#80680)
* add support for sortBy field selector

* use label selectors instead of field selectors

* set entity_labels on create & update

* make entity server integration tests work

* test fixes

* be more consistent with handling of empty body, meta or status

* workaround for database is locked errors during migration

* fix double import of sqlite3

* rename functions and tidy up

* refactor update

* disable integration tests until we can fix the database locking issue
2024-02-07 15:05:10 -05:00
67b2972052 Chore: add/update sqlstore-related helper functions (#77408)
* add/update sqlstore-related helper functions

* add documentation & tests for InsertQuery and UpdateQuery, make generated SQL deterministic by sorting columns

* remove old log line
2023-11-03 10:30:52 -04:00
edc34fc836 Instrumentation: Improve instrumentation for database migrations (#74659) 2023-09-11 14:10:21 +03:00
a6279b2d62 Database: Make dialects independent of xorm Engine (#69955)
* make dialects independent of xorm Engine

* goimports
2023-06-14 16:13:36 -04:00
26658d172f Migrations: Support independent migration log for entity api (#68511) 2023-05-22 11:31:07 -07:00
035bf29146 RBAC: Remove the option to disable RBAC and add automated permission migrations for instances that had RBAC disabled (#66652)
* RBAC: Stop reading enabeld from ini file and always set to true

* Migrations: Add a migration for rbac to reset data migrations if rbac
was disabled

* If rbac was disabled we reset the data and data migrations that rbac
  has to perform to get it to a correct state

* Migrator: Store migration logs on migrator and add function to clear it from the
in-memory stored logs

* update tests

---------

Co-authored-by: Karl Persson <kalle.persson@grafana.com>
2023-04-19 16:34:19 +01:00
55d2d872ec Chore: Missed deprecations due to overly broad lint exclusion (#59732) 2022-12-14 12:32:45 +01:00
31630edf0c Chore: Remove Wrapf (#50128)
* Chore: Remove Wrapf

* Remove all Wrapf refs

* Remove last Wrapf ref

* Fix lint errors

* Remove Wrap and Wrapf definitions

* Remove unnecessary colon
2022-06-06 22:30:31 +02:00
219e848e73 SQLStore: Prevent migration_id duplicates (#47774)
* SQLStore: Prevent migration_id duplicates

* Migrations: Remove non-executed migration (duplicated id)

Co-authored-by: Leonard Gram <leo@xlson.com>
2022-06-04 01:59:49 +02:00
3c3039f5b3 Chore: Remove Wrap (#50048)
* Chore: Remove Wrap and Wrapf

* Fix: Add error check
2022-06-03 09:24:24 +02:00
d718ee1918 SQLStore: Prevent concurrent migrations (#44101)
* SQLStore: Prevent concurrent migrations

* Hide behind a feature toggle

* Configurable locking attempt timeout

* Update docs/sources/administration/configuration.md

Co-authored-by: Igor Suleymanov <radiohead@users.noreply.github.com>
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
2022-02-15 18:54:27 +02:00
6523486122 Alerting: Make Unified Alerting enabled by default for those who do not use legacy alerting (#42200)
* update AlertingEnabled and UnifiedAlertingSettings.Enabled to be pointers
* add a pseudo migration to fix the AlertingEnabled and UnifiedAlertingSettings.Enabled if the latter is not defined
* update the default configuration file to make default value for both 'enabled' flags be undefined

Misc
* update Migrator to expose DB engine. This is needed for a ualert migration to access the database while the list of migrations is created.
* add more verbose failure when migrations do not match

Co-authored-by: gotjosh <josue@grafana.com>
Co-authored-by: Yuriy Tseretyan <yuriy.tseretyan@grafana.com>
Co-authored-by: gillesdemey <gilles.de.mey@gmail.com>
2021-11-24 14:56:07 -05:00
04d5dcb7c8 Alerting: modify DB table, accessors and migration to restrict org access (#37414)
* Alerting: modify table and accessors to limit org access appropriately

* Update migration to create multiple Alertmanager configs

* Apply suggestions from code review

Co-authored-by: gotjosh <josue@grafana.com>

* replace mg.ClearMigrationEntry()

mg.ClearMigrationEntry() would create a new session.
This commit introduces a new migration for clearing an entry from migration log for replacing  mg.ClearMigrationEntry() so that all dashboard alert migration operations will run inside the same transaction.
It adds also `SkipMigrationLog()` in Migrator interface for skipping adding an entry in the migration_log.

Co-authored-by: gotjosh <josue@grafana.com>
2021-08-12 16:04:09 +03:00
423474325a Alerting: fix postgres migration on conflict (#35556) 2021-06-14 16:43:33 +03:00
dc3e17ba5b Alerting: Change rule migration to be based on feature flag (#33792)
makes it so the feature flag can be turned on off, and the migration will be cleared and rerun. All existing NG alert rules, configuration settings, etc are removed when disabling the feature flag.
for https://github.com/grafana/alerting-squad/issues/142

Co-authored-by: Sofia Papagiannaki <sofia@grafana.com>
2021-05-11 08:08:39 -04:00
0c2bcbf2bc Live: persisting last message in cache for broadcast scope (#32938) 2021-04-30 11:06:33 -07:00
c899bf3592 log skipped, performed and duration for migrations (#31722)
Signed-off-by: bergquist <carl.bergquist@gmail.com>
2021-03-07 21:26:29 +01:00
25048ebdf8 Chore: Add CloudWatch HTTP API tests (#29691)
* CloudWatch: Add HTTP API tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-01-07 11:36:13 +01:00
3cac10e598 AlertingNG: Create a scheduler to evaluate alert definitions (#29305)
* Always use cache: stop passing skipCache among ngalert functions

* Add updated column

* Scheduler initial draft

* Add retry on failure

* Allow settting/updating alert definition interval

Set default interval if no interval is provided during alert definition creation.
Keep existing alert definition interval if no interval is provided during alert definition update.

* Parameterise alerting.Ticker to run on custom interval

* Allow updating alert definition interval without having to provide the queries and expressions

* Add schedule tests

* Use xorm tags for having initialisms with consistent case in Go

* Add ability to pause/unpause the scheduler

* Add alert definition versioning

* Optimise scheduler to fetch alert definition only when it's necessary

* Change MySQL data column to mediumtext

* Delete alert definition versions

* Increase default scheduler interval to 10 seconds

* Fix setting OrgID on updates

* Add validation for alert definition name length

* Recreate tables
2020-12-17 16:00:09 +02:00
9593d57914 Chore: Enable errorlint linter (#29227)
* Enable errorlint linter
* Handle wrapped errors

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2020-11-19 14:47:17 +01:00
b5379c5335 Chore: Fix SQL related Go variable naming (#28887)
* Chore: Fix variable naming

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-11-11 06:21:08 +01:00
102448040d SQLite: Set 0640 permissions on SQLite database file (#26339)
* SQLite: Set 640 permissions on SQLite database file
2020-07-23 15:47:26 +02:00
d1e6214a4a Chore: Enable scopelint Go linter (#25896)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-06-29 17:04:38 +02:00
ea35ae4d1f Chore: Update the xorm dependency (#22376)
* Upgrade xorm

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-04-01 15:57:21 +02:00
2a78d2a61c pkg/services: Check errors (#19712)
* pkg/services: Check errors
* pkg/services: Don't treat context.Canceled|context.DeadlineExceeded as error
2019-10-22 14:08:18 +02:00
2fff8f77dc move log package to /infra (#17023)
ref #14679

Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-05-13 08:45:54 +02:00
12f3c8f9d6 fix only add column if not exists for mysql 2018-12-18 23:02:08 +01:00
09b3014683 fix only create/drop database indices if not exists/exists 2018-12-18 21:47:45 +01:00
92ed1f04af sql: added code migration type 2018-08-21 13:53:04 +02:00
6c0752473a Sqlstore refactor (#11908)
* refactor: tracing service refactoring

* refactor: sqlstore to instance service

* refactor: sqlstore & registory priority

* refactor: sqlstore refactor wip

* sqlstore: progress on getting tests to work again

* sqlstore: progress on refactoring and getting tests working

* sqlstore: connection string fix

* fix: not sure why this test is not working and required changing expires

* fix: updated grafana-cli
2018-05-18 11:10:10 +02:00
1dfff74da9 move database-specific code into dialects (#11884) 2018-05-10 16:54:21 +02:00
de8696d5d3 Outdent code after if block that ends with return (golint)
This commit fixes the following golint warnings:
pkg/bus/bus.go:64:9: if block ends with a return statement, so drop this else and outdent its block
pkg/bus/bus.go:84:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:137:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:177:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:183:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:199:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:208:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/components/dynmap/dynmap.go:236:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:242:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:257:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:263:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:278:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:284:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:299:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:331:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:350:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:356:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:366:12: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:390:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:396:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:405:12: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:427:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:433:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:442:12: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:459:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:465:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:474:12: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:491:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:497:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:506:12: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:523:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:529:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:538:12: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:555:9: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:561:10: if block ends with a return statement, so drop this else and outdent its block
pkg/components/dynmap/dynmap.go:570:12: if block ends with a return statement, so drop this else and outdent its block
pkg/login/ldap.go:55:11: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/login/ldap_test.go:372:10: if block ends with a return statement, so drop this else and outdent its block
pkg/middleware/middleware_test.go:213:12: if block ends with a return statement, so drop this else and outdent its block
pkg/plugins/dashboard_importer.go:153:11: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/plugins/dashboards_updater.go:39:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/plugins/dashboards_updater.go:121:10: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/plugins/plugins.go:210:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/plugins/plugins.go:235:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/alerting/eval_context.go:111:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/alerting/notifier.go:92:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/alerting/notifier.go:98:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/alerting/notifier.go:122:10: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/alerting/rule.go:108:10: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/alerting/rule.go:118:10: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/alerting/rule.go:121:11: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/alerting/notifiers/telegram.go:94:10: if block ends with a return statement, so drop this else and outdent its block
pkg/services/sqlstore/annotation.go:34:11: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/sqlstore/annotation.go:99:11: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/sqlstore/dashboard_test.go:107:13: if block ends with a return statement, so drop this else and outdent its block
pkg/services/sqlstore/plugin_setting.go:78:10: if block ends with a return statement, so drop this else and outdent its block
pkg/services/sqlstore/preferences.go:91:10: if block ends with a return statement, so drop this else and outdent its block
pkg/services/sqlstore/user.go:50:10: if block ends with a return statement, so drop this else and outdent its block
pkg/services/sqlstore/migrator/migrator.go:106:11: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/services/sqlstore/migrator/postgres_dialect.go:48:10: if block ends with a return statement, so drop this else and outdent its block
pkg/tsdb/time_range.go:59:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/tsdb/time_range.go:67:9: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
pkg/tsdb/cloudwatch/metric_find_query.go:225:9: if block ends with a return statement, so drop this else and outdent its block
pkg/util/filepath.go:68:11: if block ends with a return statement, so drop this else and outdent its block (move short variable declaration to its own line if necessary)
2018-04-27 23:10:43 +02:00
1c20126f87 database: update xorm to v0.6.4 and xorm core to v0.5.7 2018-03-16 00:02:50 +01:00
56907eef69 tests: makes sure we all migrations are working 2018-02-14 15:30:29 +01:00
5ec86a9ef4 fix(build): fixed broken build, unreachable code in migrator.go 2016-09-30 09:23:26 +02:00
2b8177e3e5 fix(migration): fix for sqlstore migration, the execution of the migration and recording of the success of it was not done in same transaction, fixes #5315 2016-09-30 07:49:40 +02:00
086b59483e feat(loggin): disable logging by default so unit tests are not full of logging 2016-06-11 11:38:25 +02:00
a02cf5beb7 feat(logging): added loging filters 2016-06-07 12:11:41 +02:00
22778e6efd feat(logging): a lot of progress on moving to new logging lib, #4590 2016-06-06 23:06:44 +02:00
16fa5c4df3 fix(mysql): fix for migration in newly added temp_user table, fixes #2509 2015-08-14 09:41:07 +02:00
563d5e3ad8 All migrations work in sqlite3, mysql and postgres 2015-02-25 07:57:51 +01:00
02a89c752b Progress on database schema migration for account -> org refactor 2015-02-24 11:46:34 +01:00