Guillaume Michel 006f9dc704 feat: opt-in new Sweep provide system (#10834) ...

* reprovide sweep draft

* update reprovider dep

* go mod tidy

* fix provider type

* change router type

* dual reprovider

* revert to provider.System

* back to start

* SweepingReprovider test

* fix nil pointer deref

* noop provider for nil dht

* disabled initial network estimation

* another iteration

* suppress missing self addrs err

* silence empty rt err on lan dht

* comments

* new attempt at integrating

* reverting changes in core/node/libp2p/routing.go

* removing SweepingProvider

* make reprovider optional

* add noop reprovider

* update KeyChanFunc type alias

* restore boxo KeyChanFunc

* fix missing KeyChanFunc

* test(sharness): PARALLEL=1 and timeout 30m

running sequentially to see where timeout occurs

* initialize MHStore

* revert workflow debug

* config

* config docs

* merged IpfsNode provider and reprovider

* move Provider interface to from kad-dht to node

* moved Provider interface from kad-dht to kubo/core/node

* mod_tidy

* Add Clear to Provider interface

* use latest kad-dht commit

* make linter happy

* updated boxo provide interface

* boxo PR fix

* using latest kad-dht commit

* use latest boxo release

* fix fx

* fx cyclic deps

* fix merge issues

* extended tests

* don't provide LAN DHT

* docs

* restore dual dht provider

* don't start provider before it is online

* address linter

* dual/provider fix

* add delay in provider tests for dht bootstrap

* add OfflineDelay parameter to config

* remove increase number of workers in test

* improved keystore gc process

* fix: replace incorrect logger import in coreapi

replaced github.com/labstack/gommon/log with the standard
github.com/ipfs/go-log/v2 logger used throughout kubo.
removed unused labstack dependency from go.mod files.

* fix: remove duplicate WithDefault call in provider config

* fix: use correct option method for burst workers

* fix: improve error messages for experimental sweeping provider

updated error messages to clearly indicate when commands are unavailable
due to experimental sweeping provider being enabled via Reprovider.Sweep.Enabled=true

* docs: remove obsolete KeyStoreGCInterval config

removed from config.md as option no longer exists (removed in b540fba1a)
updated keystore description to reflect gc happens at reprovide interval

* docs: add TODO placeholder changelog for experimental sweeping DHT provider

using v0.38-TODO.md name to avoid merge conflicts with master branch
and allow CI tests to run. will be renamed to v0.38.md once config
migration is added to the PR

* fix: provideKeysRec go routine

* clear keystore on close

* fix: datastore prefix

* fix: improve error handling in provideKeysRec

- close errCh channel to distinguish between nil and pending errors
- check for pending errors when provided.New closes
- handle context cancellation during error send
- prevent race condition where errors could be silently lost

this ensures DAG walk errors are always propagated correctly

* address gammazero's review

* rename BurstProvider to LegacyProvider

* use latest provider/keystore

* boxo: make mfs StartProviding async

* bump boxo

* chore: update boxo to f2b4e12fb9a8ac138ccb82aae3b51ec51d9f631c

- updated boxo dependency to specified commit
- updated go.mod and go.sum files across all modules

* use latest kad-dht/boxo

* Buffered SweepingProvider wrapper

* use latest kad-dht commit

* allow no DHT router

* use latest kad-dht & boxo

---------

Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: gammazero <11790789+gammazero@users.noreply.github.com>

2025-09-18 17:22:04 +02:00

5.8 KiB

Raw Blame History

View Raw