37 Commits

Author SHA1 Message Date
757be6365a CI: Bump golangci-lint to 2.0.2 (#103572) 2025-04-10 14:42:23 +02:00
3fad6183aa CloudMigrations: Create resource dependency map to validate snapshot request (#102594)
* CloudMigrations: Create resource dependency map to validate snapshot request

* CloudMigrations: Validate resource types dependencies in create snapshot request

* CloudMigrations: Update service interface to pass parsed resource types for creation

* CloudMigrations: Conditionally append resource to snapshot if enabled

* CloudMigrations: Add /cloudmigration/resources/dependencies endpoint

* CloudMigrations: Properly filter dashboards and folders from snapshot
2025-04-08 17:46:25 +02:00
0fd7dfeb2a CloudMigrations: Implement table sorting in the UI (#103061)
* implement sorting

* swagger gen

* minor fixes

* clean up param reading

* add todo

* add errors only prop

* codegen stuff

* fix copy paste error

* forgot the api gen

* implement sorting on the frontend

* remove unused enums

* cleanup

* remove tests that are obe

* fix merge

* change page length based on checked box

* cleanup

* fix test

* i18n-extract

* var name tweaks

* use switch and conditional rendering instead of checkbox

* i18n-extract

* use initial sort

* fix type

* fix layout

* fix bugs

* make betterer happy

* fix var names
2025-04-08 08:14:17 -04:00
88e51d549c CloudMigrations: Add sorting and error filtering to Snapshot Results backend (#102753)
* implement sorting

* swagger gen

* minor fixes

* clean up param reading

* add todo

* add errors only prop

* codegen stuff

* fix copy paste error

* forgot the api gen

* cleanup

* remove tests that are obe

* fix test
2025-03-30 10:42:45 -04:00
3958fb9e0a CloudMigrations: Introduce RBAC role for migration assistant (#98588)
* CloudMigrations: delete unused code

* CloudMigrations: add access control and protect API + navtree with action

* CloudMigrations: register access control roles

* CloudMigrations: gate frontend based with access control

* CloudMigrations: fix api tests

* CloudMigrations: add docs on new actions and roles

* CloudMigrations: dont interpolate vars to make it more greppable

* CloudMigrations: run prettier
2025-01-09 06:03:42 +02:00
924b0d0863 CloudMigration: Add user uid to on prem event collection (#94804)
* add useruid to report events

* test
2024-11-13 11:07:00 +00:00
750a0bed71 MigrationAssistant: Restrict dashboards, folders and datasources by the org id of the signed in user (#96339)
apply security patch: main/206-202410241510.patch
2024-11-12 13:03:16 -06:00
a50507e645 CloudMigration: Interpret error code from migration resource item (#94407)
* start on loading the error code

* error code to message mapping

* use resource code type

* use defined error code

* partial updates from comments

* i18nKey gen

* fixed t

* fixed translations

* typing
2024-10-17 14:09:09 +01:00
d88be2819d CloudMigrations: Store parent folder name in cloud_migration_resource table (#94009)
* use name in fe

* store parent folder name in local db

* clean up

* tiny test

* trial react

* rename to parent name

* go lint

* generate api and ts

* go tests

* rearrange

* clean

* update with suggestions from josh

* make library elements work

* updates from comments

* global migration types

* parent name for alter table
2024-10-07 11:35:08 +01:00
19c77eaae1 CloudMigrations: Fix traceability & HTTP Client initialisation (#94141)
* Add traceability to Migration Assistant feature

* Fix some compilation errors

* Fix lint issues

* Use async context

* Add trace for LibraryElements
2024-10-07 11:31:45 +02:00
1a31abe254 CloudMigrations: Limit frontend query to get latest snapshots (#93639)
* latest param to endpoint and adapt frontend query

* change to sort param

* api

* remove description
2024-10-01 09:28:25 +01:00
1b7cd4c684 Cloud migrations: store resource name in the cloud_migration_resource table (#93612)
* Cloud migrations: store resource name in the cloud_migration_resource table

* remove unused function: convertMigrateDataResponseToDTO

* make swagger-clean && make openapi3-gen

* use DB_Text for cloud_migration_resource.name instead of DB_Varchar
2024-09-26 11:16:52 -03:00
945484232c CloudMigrations: remove unused code from sync migration approach (#93430) 2024-09-20 08:58:11 +02:00
6cd0971dc6 CloudMigration: Show warning message for successfully migrated non-core data sources (#91545)
* minor performance improvement

* apply a warning to any non-core plugins that successfully migrate

* commit frontend wip while I refactor some stuff

* update api

* repurpose error dialog to be a generic details dialog

* whitespace

* add unit test

* fixes from testing

* fix migration summary

* add comment

* fix localization stuff

* fix backend test

* reduce number of queries to the db

* some PR feedback

* whitespace
2024-08-06 19:35:10 +03:00
235f223aef CloudMigrations: Fix typo in trace name (#91261) 2024-07-31 10:24:40 -03:00
dc355331a6 CloudMigrations: Store encryption key in unified secrets table (#90908)
* store encryption key in unified secrets table

* fix local dev mode

* make metadata more realistic

* fix tests

* fix sql queries against postgres

* fix stats endpoint
2024-07-25 06:34:25 +03:00
317da43a84 CloudMigration: Provide a stats rollup in the GetSnapshot response (#90252)
* order session list descending

* add snapshot status method to store

* query stats while retrieving snapshot

* return stats in dto

* swagger

* fix tests

* commit results of bingo get

* fix swagger

* minor improvement

* fix typo

* forgot a file
2024-07-10 08:46:38 -04:00
d1952bb681 Cloud migrations: create snapshot files (#89693)
* Cloud migrations: create snapshot and store it on disk

* fix merge conflicts

* implement StartSnapshot for gms client

* pass snapshot directory as argument to snapshot builder

* ensure snapshot folder is set

* make swagger-gen

* remove Test_ExecuteAsyncWorkflow

* pass signed in user to buildSnapshot method / use github.com/grafana/grafana-cloud-migration-snapshot to create snapshot files

* fix FakeServiceImpl.CreateSnapshot

* remove new line
2024-07-03 10:38:26 -03:00
4d69213829 CloudMigrations: Break snapshot resources out into their own table (#89575)
* create a new table for migration resources

* remove raw result bytes from db

* more snapshot resource management stuff

* integrate new table with snapshots

* pass in result limit and offset as params

* combine create and update

* set up xorm store test

* add unit tests

* save some cpu

* remove unneeded arg

* regen swagger

* fix bug with result processing

* fix update create logic so that uid isn't required for lookup

* change offset to page

* regen swagger

* revert accidental changes to file

* curl command page should be 1 indexed
2024-06-24 23:50:07 -04:00
89337ea01f CloudMigrations: Add unit tests for snapshot management (#89521)
* add regex support for api tests

* revert dumb thing

* add api tests

* add unit test for core async workflow

* add xorm store unit tests

* fix typo

* remove unnecessary assignment
2024-06-21 16:35:15 +03:00
8a8f97b0e4 CloudMigrations: Implement snapshot management apis (#89296)
* add new apis

* add payloads

* create snapshot status type

* add some impl

* finish implementing update

* start implementing build snapshot func

* add more fake build logic

* add cancel endpoint. do some cleanup

* implement GetSnapshot

* implement upload snapshot

* merge onprem status with gms result

* get it working

* update comment

* rename list endpoint

* add query limit and offset

* add helper method to snapshot

* little bit of cleanup

* work on swagger annotations

* manual merge

* generate swagger specs

* clean up curl commands

* fix bugs found during final testing

* fix linter issue

* fix unit test
2024-06-19 09:20:52 -04:00
9d3a4e236d CloudMigrations: Refactor API for async work (#89084)
* rename some stuff

* more renaming

* clean up api

* rename more functions

* rename cms -> gms

* update comment

* update swagger gen

* update endpoints

* overzealous

* final touches

* dont modify existing migrations

* break structs into domain and dtos

* add some conversion funcs

* fix build

* update frontend

* try to make swagger happy
2024-06-13 17:58:59 +00:00
33b9544047 Cloud migrations: create route to delete token (#88297)
* Cloud migrations: create route to delete token

* gcom.DeleteToken returns ErrTokenNotFound instead of a boolean

* remove unnecessary comment

* make openapi3-gen && yarn run rtk-query-codegen-openapi ./scripts/generate-rtk-apis.ts

* gcom stub: implement DeleteToken
2024-05-31 10:03:43 -03:00
83543c6b12 Cloud migrations: create route to fetch cloud migration token (#88176)
* Cloud migration: create route to fetch cloud migration token

* implement gcomStub.ListTokens

* fix swagger for POST /cloudmigration/migration

* fix swagger for POST /cloudmigration/migration

* fix swagger for POST /cloudmigration/migration
2024-05-31 09:39:10 -03:00
14ccf7f6d1 Cloud migrations: return errutil.Error in http responses (#87448) 2024-05-13 00:22:46 -04:00
6a3ab71d77 cms: clean up model file (#87179)
* move CreateAccessTokenResponseDTO to api.go

* remove unused types

* run make swagger-clean && make openapi3-gen
2024-05-13 00:22:19 -04:00
0719f73f35 Cloud Migration: Remove ID migration in favor of UID (#86550)
* - Added migration to add a new collumn UID to both migration and migration_run tables.
- Added migration to set UID for records already existent in the database before adding a new column.
- Added UID field to CloudMigration and CloudMigrationRun dtos (model.go)

* Fix db migration

* Updating store and model

* Updating API to use UID

* fix typo

* update openapi specs and generated endpoints

* fix spec

* update openapi specs and generated endpoints

* Fixing db mapping

* Fix frontend

* fix migration

* remove migration uid from run fetch endpoint

* Revert "merge"

This reverts commit d654e4c530caebe439060c876c44132e74a28de8, reversing
changes made to 5fe0b483eb62cc9dfceb01043d2165617d974ed9.

* manual merge

* rename some funcs for consistency

* make interfaces consistent

* validate uids

* update generated frontend api

* fix enterprise spec

* manually resolve api

* try again

* try yet again

* once more

---------

Co-authored-by: Josh Hunt <joshhunt@users.noreply.github.com>
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
Co-authored-by: joshhunt <josh@trtr.co>
2024-05-01 12:29:25 -04:00
df4c8c3cbc CloudMigrations: Move business logic out of api layer (#86406)
* move run migration to the cloudmigrationimpl layer

* add migration run list logic down a layer

* remove useless comments

* pull cms calls into their own service
2024-04-17 15:43:09 -04:00
c8d185502b Fixes CMS url for dev environment (#86056) 2024-04-12 14:19:38 +00:00
1332d8ba24 Cloudmigration: Sets the runID correctly when saving (#85661)
Sets the runID correctly instead of setting it to the same as the migration id.
2024-04-05 18:03:45 +02:00
ddf5fbe591 CMS: Update permissions required to use endpoints (#85555)
* try different perms

* fix compile issue

* remove ac dependency

* remove dependency
2024-04-03 22:43:48 +03:00
a457ab3192 Cloudmigration: decode json data (#85548)
* decode get

* decode bytes

* response

* .

* linter

* quick fixes

---------

Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
2024-04-03 18:47:49 +02:00
b885da09da CloudMigrations: Implement migrations API (#85348)
* Implement run migration endpoint

* Refactor RunMigration method into separate methods

* Save migration runs fix lint

* Minor changes

* Refactor how to use cms endpoint

* fix interface

* complete merge

* add individual items

* adds tracing to getMigration

* linter

* updated swagger definition with the latest changes

* CloudMigrations: Implement core API handlers for cloud migrations and migration runs (#85407)

* implement delete

* add auth token encryption

* implement token validation

* call token validation during migration creation

* implement get migration status

* implement list migration runs

* fix bug

* finish parse domain func

* fix urls

* fix typo

* fix encoding and decoding

* remove double decryption

* add missing slash

* fix id returned by create function

* inject missing services

* finish implementing (as far as I can tell right now) data migration and response handling

* comment out broken test, needs a rewrite

* add a few final touches

* get dashboard migration to work properly

* changed runMigration to a POST

* swagger

* swagger

* swagger

---------

Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
Co-authored-by: Michael Mandrus <41969079+mmandrus@users.noreply.github.com>
2024-04-03 13:36:13 +02:00
4e89267603 Cloudmigration: swagger for the cloudmigration api (#85255)
* swagger for create token

* tried to fix errors

* update swagger annotations

* update swagger annotations

* update api GetMigrationList to return the *Response directly

* clean up previous commit

---------

Co-authored-by: joshhunt <josh@trtr.co>
2024-04-02 13:57:42 +02:00
a2e21eac8c Cloud migrations: create endpoint to create an access token (#84690)
* fix merge conflicts

* make token expiration configurable
2024-03-25 12:43:28 -03:00
e2f155f9f7 Grafana: define the api for the grafana cloudmigration api (not the csm api) (#84430)
* Add cloud migration endpoints

* Built auth into creating a migration.

* Added more detail to the migration result model

* goimports

* Update pkg/services/cloudmigration/api/api.go

Co-authored-by: lean.dev <34773040+leandro-deveikis@users.noreply.github.com>

* Update pkg/services/cloudmigration/api/api.go

Co-authored-by: lean.dev <34773040+leandro-deveikis@users.noreply.github.com>

---------

Co-authored-by: Leonard Gram <leo@xlson.com>
Co-authored-by: lean.dev <34773040+leandro-deveikis@users.noreply.github.com>
2024-03-25 13:30:47 +01:00
80447dd0cc Restructure cloudmigration service (#83211)
* Restructure cloudmigation service

* Adjust codewoners and wire

* Comment out unused metrics
2024-02-26 14:52:16 +01:00