* fix(config): explicit Provider.Enabled flag
Adds missing config option described in
https://github.com/ipfs/kubo/issues/10803
* refactor: remove Experimental.StrategicProviding
removing experiment, replaced with Provider.Enabled
* test(cli): routing [re]provide
updated and added tests for manually triggering provide and reprovide
and making them respect global configuration flag to avoid
inconsistent behaviors
* docs: improve DelegatedRouters
* refactor: default DefaultProviderWorkerCount=16
- simplified default for both
- 16 is safer for non-accelerated DHT client
- acceletated DHT performs better without limit anyway - updated docs
* Add MFS command line options, extend existing mount functions for MFS, set defaults.
* Directory listing and file stat.
* Add a read-only MFS view.
* Add mkdir and interface checks.
* Add remove and rename functionality.
* Implement all required write interfaces.
* Adjust mount functions for other architechtures.
* Merge branch 'master' into feat/10710-mfs-fuse-mount
* Write a basic read/write test.
* Write more basic tests, add a mutex to the file object, fix modtime.
* Add a concurrency test, remove mutexes from file and directory structures.
* Refactor naming(mfdir -> mfsdir) and add documentation.
* Add CID retrieval through ipfs_cid xattr.
* Add docs, add xattr listing, fix bugs for mv and stat, refactor.
* Add MFS command line options, extend existing mount functions for MFS, set defaults.
* docs phrasing
* docs: Mounts.MFS
* docs: warn about lazy-loaded DAGs
* test: TEST_FUSE=1 ./t0030-mount.sh -v
---------
Co-authored-by: Guillaume Michel <guillaumemichel@users.noreply.github.com>
Co-authored-by: guillaumemichel <guillaume@michel.id>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* adjust ipfs stats provide
* update boxo dep
* bump boxo
* fixing tests
* docs/chore: mark stat reprovide as experimental
* docs: Provider.Strategy
explicitly document it is not used - without this legacy users will have
it in their config and be very confused
---------
Co-authored-by: Marcin Rataj <lidel@lidel.org>
https://github.com/ipfs/kubo/pull/10677 removed command without properly
deprecating it first, this restores it and marks as deprecated
we can remove it after at least 1 release with deprecation being
announced
* feat: expose BlockKeyCacheSize and enable WriteThrough when bloom filter disabled
* import/config: add BatchMaxSize and BatchMaxNodes
* config: make BlockKeyCacheSize an OptionalInteger
* config: add and wire datastore.WriteThrough option
* config: omitempty on BlockKeyCacheSize
* changelog: rewrite entry about new options for the datastore
* config: add docs for BatchMaxNodes and BatchMaxSize
* config: make WriteThrough an optional Flag
* changelog: improve description of new datastore/import options
* refactor: DefaultWriteThrough as bool
* chore: boxo v0.26.0
* docs: config and changelog fixes
* cmd/files: flush parent folders
This is a mitigation to increased MFS memory usage in the course of many writes operations.
The underlying issue is the unbounded growth of the mfs directory cache in
boxo. In the latest boxo version, this cache can be cleared by calling Flush()
on the folder. In order to trigger that, we call Flush() on the parent folder
of the file/folder where the write-operations are happening.
To flushing the parent folder allows it to grow unbounded. Then, any read
operation to that folder or parents (i.e. stat), will trigger a sync-operation to match
the cache to the underlying unixfs structure (and obtain the correct node-cid).
This sync operation must visit every item in the cache. When the cache has grown too much,
and the underlying unixfs-folder has switched into a HAMT, the operation can take minutes.
Thus, we should clear the cache often and the Flush flag is a good indicator
that we can let it go. Users can always run with --flush=false and flush at
regular intervals during their MFS writes if they want to extract some performance.
Fixes#8694, #10588.
* cmd/files: docs and changelog for --flush changes
This is a cosmetic fix for bug found during testing 0.29.0-rc2.
pin add --name had shorthand -n
pin ls --name had no shorthand, and --names had -n
This unifies -n making it a shorthand for the same parameter in both
`pin ls` and `pin add`.
In that situation, the data is not written to permanent storage, so a reference in MFS would be to p2p blocks at best.
The /add command in that situation is also likely to hang as it reads immediately the root node without being able to get it (it falls back to bitswap).
this command used to work with domain without `/ipns/` prefix.
we've switched it to the same backend as `resolve` command,
which requires the prefix, so we add it if it is missing
* fix(commands): routing put command returns the IPNS ID rather than the host's ID
* fix(commands): routing put command errors with the allow-offline hint if the error is an offline error
* fix: test expects correct error message
---------
Co-authored-by: Henrique Dias <hacdias@gmail.com>
Fixes#8492
This introduces "nopfs" as a preloaded plugin into Kubo
with support for denylists from https://github.com/ipfs/specs/pull/383
It automatically makes Kubo watch *.deny files found in:
- /etc/ipfs/denylists
- $XDG_CONFIG_HOME/ipfs/denylists
- $IPFS_PATH/denylists
* test: Gateway.NoFetch and GatewayOverLibp2p
adds missing tests for "no fetch" gateways one can expose,
in both cases the offline mode is done by passing custom
blockservice/exchange into path resolver, which means
global path resolver that has nopfs intercept is not used,
and the content blocking does not happen on these gateways.
* fix: use offline path resolvers where appropriate
this fixes the problem described in
https://github.com/ipfs/kubo/pull/10161#issuecomment-1782175955
by adding explicit offline path resolvers that are backed
by offline exchange, and using them in NoFetch gateways
instead of the default online ones
---------
Co-authored-by: Henrique Dias <hacdias@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
