4a779c4ccb
fix: unified resource server list queries order column ( #108094 )
2025-07-16 08:09:31 +02:00
7002ab90ae
unistore: save returns a writecloser ( #107955 )
...
* unistore: save returns a writecloser
* go-lint
* address comments
2025-07-11 18:25:48 +02:00
ea0ddb3fc9
unistore: refactor get to return a reader ( #107951 )
2025-07-11 11:10:19 +02:00
b6dd08da2f
unistore: fix delete and db closed in kv store ( #107918 )
...
* fix delete and db closed
* fix tests
2025-07-10 11:34:36 +02:00
9d2eadcfd2
PoC: replace using testing.Short in storage/unified package with integration test skip method ( #107887 )
...
* PoC: check if testing.Short is called from integration tests only.
* Rename helper function.
* Fix logic.
* Remove skipping of integration tests from non-integration tests.
* Remove skipping of integration tests from non-integration tests.
* Fix import.
2025-07-09 15:32:10 +00:00
7787ff6b2e
unistore: add more kvstore tests ( #107692 )
...
add more kvstore tests
2025-07-07 15:00:40 +02:00
8d8b824f73
unistore: skipping badger test failing atm ( #107572 )
...
skipping badger test failing atm
2025-07-03 12:31:07 +00:00
d55541735a
unistore: use different ns for each test ( #107509 )
...
use different ns for each test
2025-07-02 13:50:30 +02:00
696657bdd1
unistore: add kv based storage backend ( #107305 )
...
* Add datastore
* too many slashes
* lint
* add metadata store
* simplify meta
* Add eventstore
* golint
* lint
* Add datastore
* too many slashes
* lint
* pr comments
* extract ParseKey
* readcloser
* remove get prefix
* use dedicated keys
* parsekey
* sameresource
* unrelated
* name
* renmae tests
* add key validation
* fix tests
* refactor a bit
* lint
* allow empty ns
* get keys instead of list
* rename the functions
* refactor yield candidate
* update test
* unistore: add LastResourceVersion to datastore
* lint
* use map string
* missing err check
* fix
* Add storage backend
* remove hasmore
* fix tests
* small refactor
* pre-alloc
* extract the folder
* lint
* refactor
* handle context canceled in ListHistory to pass the tests
* fix the resource test
* unistore: provide generic tests for the kv interface (#107443 )
unistore: move the kv tests to the testing package
* Update pkg/storage/unified/resource/storage_backend_test.go
Co-authored-by: Peter Štibraný <pstibrany@gmail.com>
* address comments
* comments
* comments
* comments
* normalise the names and add helper method
* events comments
* rename function
---------
Co-authored-by: Peter Štibraný <pstibrany@gmail.com>
2025-07-02 10:57:37 +00:00
0982cfd9a0
Unified Storage/Search: Add max count config for indexing ( #107255 )
...
* Add max count config for indexing
* Build empty index when max count is exceeded
* Address linting
* Refactor buildIndexes
* Add test for max count threshold
* Update test doc comments
* Refactor TestBuildIndexes_MaxCountThreshold to not use mock framework
* Rename mocks used in TestBuildIndexes_MaxCountThreshold
* Refactor mockResourceIndex
* Test setting of indexing threshold configs
* Tweak comments, log
* Fix logging in buildEmptyIndex
* Export and reuse TestDocumentBuilderSupplier
* Reuse MockResourceIndex
2025-06-27 14:00:39 +02:00
5150ebfba8
unistore: index search references ( #106954 )
2025-06-24 14:54:07 -05:00
9062d88ea0
unistore: filter trash requests ( #106767 )
...
* deleted by user
* use the correct checker
* add tests
* refactor
2025-06-24 10:17:34 +02:00
7864c1660f
K8s: Get trash fixes ( #106411 )
2025-06-11 13:54:38 -05:00
eb8f3bc071
fix(unified-storage): wait for result to be available ( #105922 )
2025-05-23 11:03:01 +02:00
a13fe75de8
refactor(unified-storage): set the GUID in the resource server ( #105683 )
2025-05-21 09:49:49 +02:00
002f46736a
refactor(unified-storage): move generated protos to own pkg ( #105356 )
2025-05-15 21:36:52 +02:00
e162c69c34
search: fix document missing at startup ( #105198 )
...
* fix document missing at startup
* go-lint
* fix tests
* fix tests
* fix integration tests now that we are storing real values
2025-05-12 07:56:25 +02:00
15b3de5893
search: add index batching ( #104163 )
...
* add basic search backend integration tests
* add search backend benchmark
* add benchmark indexServer
* fix
* lint
* add more tests
* lint
* do not use the poller
* batch write
* refactor and add tests
* improvements
* improvements
* cleanup
* only observe index success
* add monitorIndexEvents method
* nit use switch instead of if
* make newIndexQueueProcessor private
* simplify runProcessor
* go lint
2025-05-09 15:36:21 +02:00
b3b92e2471
Storage: Change timeout for flaky test ( #104065 )
...
Change timeout for flaky test
Co-authored-by: Marco de Abreu <18629099+marcoabreu@users.noreply.github.com>
2025-04-15 19:28:27 +01:00
e69052a417
unistore: add check when update the folder of a resource ( #102699 )
...
* Add check for move folder
* make the server test generic
* address comment
2025-04-14 16:57:40 +02:00
be696dd70c
unified storage: check for iterator errors after each call to iter.Next. ( #102804 )
...
* unified storage: check for iterator errors after each call to iter.Next.
* Extracted test to separate method, add 500 events before listing.
2025-04-11 16:25:40 +02:00
f0a6327edc
Unified Storage: Don't read before create ( #102906 )
...
* Unified Storage: Don't read before create
* test: use the existing test infra
* fix: support pq
We use pgx, but it seems to be wrapped in a pq driver shim, causing the errors to be remapped to pq's type. Weird
situation.
* feat: support CDK backend
* revert: there is a postgres_tests block
* fix(CDK): only check existence on ADDED updates
* fix(CDK): use ReadResource to deal with deleted files
2025-03-31 15:06:31 +02:00
fe1a72e41b
fix(unified-storage): unlist the deleted resource's versions ( #102588 )
2025-03-27 18:34:37 +01:00
6d570db312
Storage: Add ascending order support for NotOlderThan queries and introduce ResourceVersionMatch_Unset as default ( #102505 )
...
* Add support for ASC ordering and introduce ResourceVersionMatch_Unset as default
Add SortAscending to continue token and add integration test for pagination.
* Change protobuf order
* Make backwards compatible
* Update pkg/storage/unified/sql/backend.go
Co-authored-by: Jean-Philippe Quéméner <JohnnyQQQQ@users.noreply.github.com>
---------
Co-authored-by: Marco de Abreu <18629099+marcoabreu@users.noreply.github.com>
Co-authored-by: Jean-Philippe Quéméner <JohnnyQQQQ@users.noreply.github.com>
2025-03-21 18:35:32 +02:00
0bafd4e99d
unistore: update spanner ddl to include unistore tables ( #102198 )
...
* provision unistore tables
* update the tests to use generated namespace
* update ddl
* update ddl
* clean streams
* missing space
* use engine.db.query
2025-03-18 14:37:11 +01:00
3589baac68
Unistore: Batch write events ( #101381 )
...
* Batch write events
* Improve instrumentation
* Measure batch phases
* Detect lock contention
* remove the execBatch goroutine
* removing tracing prefix
* detect context cancel
* batch channel map
2025-03-13 10:24:12 +02:00
8c935c8f4a
[unistore] Add benchmark for write throughput ( #101345 )
...
* Add generic benchmark
* address comments
2025-02-26 17:17:35 +02:00
8f37822050
Unified/Blob: Allow direct access to UIDs ( #101339 )
2025-02-26 16:18:59 +02:00
53e91fd5e8
unistore: close event stream on context cancelation ( #101293 )
...
* add tests for broacaster
* fix sql notifier not closing the stream
* fix sql notifier not closing the stream
* close sub
* fix broadcaster test
* fix broadcaster test
* suggestion
2025-02-25 08:28:31 -08:00
e13bd52da6
unistore: move continue to the resource package ( #101206 )
...
* minor improvements to tests
* move continue token
* update sql backend
2025-02-24 18:02:30 +02:00
40d1677265
unistore: improve storage backend integration tests ( #101134 )
...
* Add Stats tests
* Add integration tests for getstats
* fix signature
2025-02-21 15:06:23 +01:00
6e5de80ef2
unistore: refactor the storage backend test suite ( #101061 )
...
* refactor the storage backend test suite
* move testing
* pass a newBackendFunc
2025-02-20 16:43:26 +01:00
