1
0
mirror of https://github.com/ipfs/kubo.git synced 2025-06-30 09:59:13 +08:00

5195 Commits

Author SHA1 Message Date
rht
b4a3854151 Remove chunk channels
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
2016-01-12 08:22:55 -08:00
a961b1f7ac ipfs files ls without -l is faster
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
rht
bf26b59c37 s/\[0:/\[:/g
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
2016-01-12 08:22:55 -08:00
rht
743f3edcbb strings.Split -> path.SplitList
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
2016-01-12 08:22:55 -08:00
rht
ffd859232d Replace strings.Join(elms, "/") with path.Join(elms)
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
2016-01-12 08:22:55 -08:00
rht
8abb12e760 Add config option for flatfs no-sync
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
2016-01-12 08:22:55 -08:00
bf955f3560 add closenotify and large timeout to gateway
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
18099abb93 if bucket doesnt have enough peers, grab more elsewhere
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
5e99be9899 comment multiple dagstore error checking
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
efac042e82 rework editor creation and finalization
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
d8ee7dffc3 improves memory usage of add
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
b6f28dad1b Add a --pin option to ipfs add (allowing --pin=false)
Implements a solution for #1908

This PR replaces #1909

License: MIT
Signed-off-by: Andrew Chin <achin@eminence32.net>
2016-01-12 08:22:55 -08:00
rht
86901aff3c Move parts of ipfs add into core/coreunix
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
2016-01-12 08:22:55 -08:00
38fab91013 address comments from CR
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
6faa70ee59 implement ipfs files command
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
13ef17ef16 fixup comments
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
78a8088410 Refactor ipnsfs into a more generic and well tested mfs
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
e49e610b07 p2p/net/filter: Guard with a mutex
Callers assume this is safe to call whenever, let's make it so.

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
2016-01-12 08:22:55 -08:00
6e126ed79d fuse/ipns: Only change file size in Setattr if asked to
This used to cause files e.g. being edited with `vi` to become 0-size.

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
2016-01-12 08:22:55 -08:00
fbd9cabd93 fuse/ipns, fuse/readonly: Let the fuse library set defaults for Attr
Without this, all entries will have nlink==0, which confuses a bunch
of tools. Most dramatically, systemd-nspawn enters a busy loop in its
lock utility function.

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
2016-01-12 08:22:55 -08:00
1d21fad697 fuse/readonly: Fix importer.BuildTrickleDagFromReader call
Last argument was dropped in ffd4c3f4db4be0c9e36c1645fd1b5a6c8e0d8b01

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
2016-01-12 08:22:55 -08:00
c6ee2058d9 comments from CR
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
45d4b1a8bc fixup datastore interfaces
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:22:55 -08:00
3ffebd942f Record datastore metrics for non-default datastores
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
2016-01-12 08:22:55 -08:00
8f2d820412 S3 datastore support
To test it, set up an S3 bucket (in an AWS region that is not US
Standard, for read-after-write consistency), run `ipfs init`, then
edit `~/.ipfs/config` to say

      "Datastore": {
        "Type": "s3",
        "Region": "us-west-1",
        "Bucket": "mahbukkit",
        "ACL": "private"
      },

with the right values. Set `AWS_ACCESS_KEY_ID` and
`AWS_SECRET_ACCESS_KEY` in the environment and you should be able to
run `ipfs add` and `ipfs cat` and see the bucket be populated.

No automated tests exist, unfortunately. S3 is thorny to simulate.

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
2016-01-12 08:22:55 -08:00
1174aab86b thirdparty/s3-datastore: Let caller set ACL, change default to safer "private"
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
2016-01-12 08:22:55 -08:00
33d41285d6 thirdparty/s3-datastore: Datastore keys can be binary, hex encode them for S3
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
2016-01-12 08:22:55 -08:00
9497e26fbd Implement pluggable Datastore types, with nothing implemented yet
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
2016-01-12 08:22:55 -08:00
b76581d6c7 fsrepo: Refactor to extract datastore internals
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
2016-01-12 08:22:55 -08:00
dc73744082 fsrepo: Detect uninitialized repo by missing config file
Earlier, it also checked checked the leveldb directory. That part
added no crash safety to the application, and just hardcoded
assumptions about the datastore.

If anything, this should rely on the absolute last item created by
fsrepo.Init, and there should be fsync guarantees about ordering.

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
2016-01-12 08:22:55 -08:00
6996ce6758 Remove Config file section "Datastore", it's not used
This gives us a clean slate for the new code, avoiding leftovers.

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
2016-01-12 08:22:55 -08:00
2c4a9ab7e4 fsrepo/serialize tests: Stop assuming internals of Config
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
2016-01-12 08:22:55 -08:00
d83644628b sharness: Stop assuming leveldb Datastore
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
2016-01-12 08:22:55 -08:00
4a33683443 core tests: Stop assuming internals of Config
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
2016-01-12 08:22:55 -08:00
a5f9832937 gofmt generated assets
The generated file went through some changes because of differing
go-bindata versions.

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
2016-01-12 08:22:55 -08:00
e431f35a0c update multistream naming of lazyconn
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:21:13 -08:00
2c4eb60961 allow multistream to have zero rtt stream opening
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:21:13 -08:00
859de51406 dont use searchset for indirect pin checking
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:21:13 -08:00
879cfeeec9 pin rm fails appropriately for indirect pins
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:21:13 -08:00
16c4d8cdcb addressing comments from CR
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:21:13 -08:00
5123857c4b break up GC logic
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:21:13 -08:00
b12ee40aba implement mark and sweep GC
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

dont GC blocks used by pinner

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

comment GC algo

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

add lock to blockstore to prevent GC from eating wanted blocks

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

improve FetchGraph

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

separate interfaces for blockstore and GCBlockstore

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

reintroduce indirect pinning, add enumerateChildren dag method

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:21:13 -08:00
27f34b4311 renamed {R,}Lock -> {Pin,GC}Lock
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
2016-01-12 08:21:13 -08:00
7b675e870c lock blockstore for pin add
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:21:13 -08:00
1a03d5e37a move locking out of GC branch
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:21:13 -08:00
dc3b9ed140 address concerns from PR
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:21:13 -08:00
b9e5cfaa15 merkledag FetchGraph and EnumerateChildren
This commit improves (fixes) the FetchGraph call for recursively
fetching every descendant node of a given merkledag node. This operation
should be the simplest way of ensuring that you have replicated a dag
locally.

This commit also implements a method in the merkledag package called
EnumerateChildren, this method is used to get a set of the keys of every
descendant node of the given node. All keys found are noted in the
passed in KeySet, which may in the future be implemented on disk to
avoid excessive memory consumption.

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:21:13 -08:00
da0d48e6c5 Add locking interface to blockstore
The addition of a locking interface to the blockstore allows us to
perform atomic operations on the underlying datastore without having to
worry about different operations happening in the background, such as
garbage collection.

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:21:13 -08:00
de50b21562 using multistream muxer
* ID service stream
* make the relay service use msmux
* fix nc tests

Note from jbenet: Maybe we should remove the old protocol/muxer
and see what breaks. It shouldn't be used by anything now.

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
2016-01-12 08:21:13 -08:00
4f34e0ec13 buffer msgio
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
2016-01-12 08:21:13 -08:00