31 Commits

Author SHA1 Message Date
b4fc33ff6b Chore: Fix cloud migration test when running against postgres (#105636)
make postgres integration tests happy
2025-05-19 22:59:31 +02:00
6205e126cc CloudMigrations: Fix issues with snapshot resource limits (#105425)
* fix bulk inserts

* commit progress so cursor doesn't sabotage me

* add more tests

* get everything working

* rename variable

* update comment

* regen mocks, fix k8s list method maybe

* fix bug with duplicate entries

* lint

* Snapshots: Use slices.Chunk for batching inserts

* remove extra linebreak

---------

Co-authored-by: Matheus Macabu <macabu.matheus@gmail.com>
2025-05-19 11:37:22 -04:00
652c374c4c CloudMigrations: Make table sort case insensitive (#103898)
* case insensitive sort

* fix for all db types

* add comment

* add unit test

* add a TODO to fix later
2025-04-11 20:29:07 +01:00
88e51d549c CloudMigrations: Add sorting and error filtering to Snapshot Results backend (#102753)
* implement sorting

* swagger gen

* minor fixes

* clean up param reading

* add todo

* add errors only prop

* codegen stuff

* fix copy paste error

* forgot the api gen

* cleanup

* remove tests that are obe

* fix test
2025-03-30 10:42:45 -04:00
5917ed8227 Hackaton: Add more unit tests, take 4 (#101704)
* cloudmigration/cloudmigrationimpl: run integration tests in parallel

* tsdb/mysql: run tests with actual service and instance manager

* pluginsintegration/angulardetectorsprovider: reduce job interval in test

* util: extract test that should only be ran with -race enabled and unskip it
2025-03-07 14:06:47 +01:00
c9d22f06c3 CloudMigrations: Bulk update local resources (#96002)
* wip

* make tests pass

* get all tests passing

* fixes

* some small cleanup

* fix test

* convert delimiter keys to struct keys

* dont execute empty sql statement

* remove printlns

* fix unit test

* a bit more cleanup

* whoops
2025-01-10 14:42:18 -05:00
750a0bed71 MigrationAssistant: Restrict dashboards, folders and datasources by the org id of the signed in user (#96339)
apply security patch: main/206-202410241510.patch
2024-11-12 13:03:16 -06:00
1a31abe254 CloudMigrations: Limit frontend query to get latest snapshots (#93639)
* latest param to endpoint and adapt frontend query

* change to sort param

* api

* remove description
2024-10-01 09:28:25 +01:00
1c648fd010 Chore: Fix flaky cloud migration test (#94035)
* attempt to fix flaky test

* remove skip from test
2024-10-01 03:24:47 +01:00
8f66b7fdb0 Cloud migrations: disable flaky test (#93921)
* Cloud migrations: disable flaky test

* Update pkg/services/cloudmigration/cloudmigrationimpl/xorm_store_test.go

Co-authored-by: Kevin Minehart <5140827+kminehart@users.noreply.github.com>

---------

Co-authored-by: Kevin Minehart <5140827+kminehart@users.noreply.github.com>
2024-09-27 16:36:08 +01:00
945484232c CloudMigrations: remove unused code from sync migration approach (#93430) 2024-09-20 08:58:11 +02:00
f4c5c5c418 Chore: Fix flaky cloud migration test due to ordered steps running in parallel (#93491)
don't run test in parallel
2024-09-19 18:45:02 +03:00
4f21ecf982 CloudMigrations: improve nil handling (#93257)
* CloudMigrations: fail token decryption if session is not found or without a token

* CloudMigrations: do not report event if session is nil
2024-09-17 08:59:47 +02:00
90b44fdcbf Cloud migrations: GetSnapshotList only returns snapshots that belong to a session that exists (#91798)
* Cloud migrations: GetSnapshotList only returns snapshots that belong to a session that exists

* Cloud migrations: test GetSnapshotList

* add one more test case for GetSnapshotList

* fix test

* store encryption key for testing
2024-08-13 09:57:53 -03:00
e20c7342a9 CloudMigration: Removes snapshot and resources when deleting a session (#91548) 2024-08-09 16:42:45 +03:00
3e138449bb CloudMigrations: Fix bugs found during local testing (#91163)
* send dashboard commands instead of dashboards

* move status updates before goroutine to ensure frontend polls

* fix syncing issues between snapshot state and resources

* make sessionUid a requirement for modifying snapshots

* move the function I meant to move earlier

* remove accidental commit

* another accidental commit

* verify UpdateSnapshot is called with sessionUid

* revert

* pass in session uid everywhere

* forgot to save

* fix unit test

* fix typo

* tiny tweak
2024-07-30 09:02:41 -04:00
dc355331a6 CloudMigrations: Store encryption key in unified secrets table (#90908)
* store encryption key in unified secrets table

* fix local dev mode

* make metadata more realistic

* fix tests

* fix sql queries against postgres

* fix stats endpoint
2024-07-25 06:34:25 +03:00
6ff21726b7 CloudMigrations: Query GMS for snapshot status with a results offset (#90453)
* query GMS for status with an offset

* remove unused state
2024-07-16 09:04:21 -04:00
542a1bf3ac CloudMigrations: Query Grafana Migration Status for status while the snapshot is in the cloud (#90314)
* implement querying gms for snapshot status

* add some documentation

* provide snapshot resources after snapshot is created

* add rate limiting to backend

* fix compilation error

* fix typo

* add unit tests

* finish merge

* lint

* swagger gen

* more testing

* remove duplicate test

* address a couple PR comments

* update switch statement to a map

* add timeouts to gms client through the http client

* remove extra whitespace

* put method back where it was so the PR is less confusing

* fix tests

* add todo

* fix final unit test
2024-07-15 09:22:57 -04:00
317da43a84 CloudMigration: Provide a stats rollup in the GetSnapshot response (#90252)
* order session list descending

* add snapshot status method to store

* query stats while retrieving snapshot

* return stats in dto

* swagger

* fix tests

* commit results of bingo get

* fix swagger

* minor improvement

* fix typo

* forgot a file
2024-07-10 08:46:38 -04:00
4d69213829 CloudMigrations: Break snapshot resources out into their own table (#89575)
* create a new table for migration resources

* remove raw result bytes from db

* more snapshot resource management stuff

* integrate new table with snapshots

* pass in result limit and offset as params

* combine create and update

* set up xorm store test

* add unit tests

* save some cpu

* remove unneeded arg

* regen swagger

* fix bug with result processing

* fix update create logic so that uid isn't required for lookup

* change offset to page

* regen swagger

* revert accidental changes to file

* curl command page should be 1 indexed
2024-06-24 23:50:07 -04:00
89337ea01f CloudMigrations: Add unit tests for snapshot management (#89521)
* add regex support for api tests

* revert dumb thing

* add api tests

* add unit test for core async workflow

* add xorm store unit tests

* fix typo

* remove unnecessary assignment
2024-06-21 16:35:15 +03:00
8a8f97b0e4 CloudMigrations: Implement snapshot management apis (#89296)
* add new apis

* add payloads

* create snapshot status type

* add some impl

* finish implementing update

* start implementing build snapshot func

* add more fake build logic

* add cancel endpoint. do some cleanup

* implement GetSnapshot

* implement upload snapshot

* merge onprem status with gms result

* get it working

* update comment

* rename list endpoint

* add query limit and offset

* add helper method to snapshot

* little bit of cleanup

* work on swagger annotations

* manual merge

* generate swagger specs

* clean up curl commands

* fix bugs found during final testing

* fix linter issue

* fix unit test
2024-06-19 09:20:52 -04:00
9d3a4e236d CloudMigrations: Refactor API for async work (#89084)
* rename some stuff

* more renaming

* clean up api

* rename more functions

* rename cms -> gms

* update comment

* update swagger gen

* update endpoints

* overzealous

* final touches

* dont modify existing migrations

* break structs into domain and dtos

* add some conversion funcs

* fix build

* update frontend

* try to make swagger happy
2024-06-13 17:58:59 +00:00
330da7916d Chore: Add unit test for cloudmigration package (#88868) 2024-06-07 13:46:19 -03:00
b885da09da CloudMigrations: Implement migrations API (#85348)
* Implement run migration endpoint

* Refactor RunMigration method into separate methods

* Save migration runs fix lint

* Minor changes

* Refactor how to use cms endpoint

* fix interface

* complete merge

* add individual items

* adds tracing to getMigration

* linter

* updated swagger definition with the latest changes

* CloudMigrations: Implement core API handlers for cloud migrations and migration runs (#85407)

* implement delete

* add auth token encryption

* implement token validation

* call token validation during migration creation

* implement get migration status

* implement list migration runs

* fix bug

* finish parse domain func

* fix urls

* fix typo

* fix encoding and decoding

* remove double decryption

* add missing slash

* fix id returned by create function

* inject missing services

* finish implementing (as far as I can tell right now) data migration and response handling

* comment out broken test, needs a rewrite

* add a few final touches

* get dashboard migration to work properly

* changed runMigration to a POST

* swagger

* swagger

* swagger

---------

Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
Co-authored-by: Michael Mandrus <41969079+mmandrus@users.noreply.github.com>
2024-04-03 13:36:13 +02:00
Jo
5340a6e548 Auth: Extended JWT client for OBO and Service Authentication (#83814)
* reenable ext-jwt-client

* fixup settings struct

* add user and service auth

* lint up

* add user auth to grafana ext

* fixes

* Populate token permissions

Co-authored-by: jguer <joao.guerreiro@grafana.com>

* fix tests

* fix lint

* small prealloc

* small prealloc

* use special namespace for access policies

* fix access policy auth

* fix tests

* fix uncalled settings expander

* add feature toggle

* small feedback fixes

* rename entitlements to permissions

* add authlibn

* allow viewing the signed in user info for non user namespace

* fix invalid namespacedID

* use authlib as verifier for tokens

* Update pkg/services/authn/clients/ext_jwt.go

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>

* Update pkg/services/authn/clients/ext_jwt_test.go

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>

* fix parameter names

* change asserts to normal package

* add rule for assert

* fix ownerships

* Local diff

* test and lint

* Fix test

* Fix ac test

* Fix pluginproxy test

* Revert testdata changes

* Force revert on test data

---------

Co-authored-by: gamab <gabriel.mabille@grafana.com>
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
2024-04-02 17:45:15 +02:00
d2ca95d304 Update plugin SDK to v0.219.0 (#85438) 2024-04-02 10:22:24 +02:00
383ebb2bc4 Cloudmigration: create migration (#85386)
* Cloudmigration: create migration

* fix drone build

* lint fix

* fix unit test

---------

Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
2024-03-29 01:55:27 +01:00
5b147d0847 CloudMigration: Add service to list all migrations (#85308) 2024-03-28 08:50:31 -03:00
80447dd0cc Restructure cloudmigration service (#83211)
* Restructure cloudmigation service

* Adjust codewoners and wire

* Comment out unused metrics
2024-02-26 14:52:16 +01:00