opensource/kubo
1
0
Fork 0
mirror of https://github.com/ipfs/kubo.git synced 2025-06-28 17:03:58 +08:00
Code Issues Projects Releases Wiki Activity
5,300 Commits 210 Branches 199 Tags
Commit Graph

5300 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jeromy
1efbc79223 use mfs for adds 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:22:55 -08:00
Jeromy
4fdfbc7d32 compute add size in background to not stall add operation 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:22:55 -08:00
Jeromy
dc2e343a99 add option to disable flushing files structure on writes 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:22:55 -08:00
rht
89ba9942c5 Simplify BalancedLayout test 
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
 2016-01-12 08:22:55 -08:00
rht
b4a3854151 Remove chunk channels 
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
 2016-01-12 08:22:55 -08:00
Jeromy
a961b1f7ac ipfs files ls without -l is faster 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:22:55 -08:00
rht
bf26b59c37 s/\[0:/\[:/g 
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
 2016-01-12 08:22:55 -08:00
rht
743f3edcbb strings.Split -> path.SplitList 
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
 2016-01-12 08:22:55 -08:00
rht
ffd859232d Replace strings.Join(elms, "/") with path.Join(elms) 
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
 2016-01-12 08:22:55 -08:00
rht
8abb12e760 Add config option for flatfs no-sync 
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
 2016-01-12 08:22:55 -08:00
Jeromy
bf955f3560 add closenotify and large timeout to gateway 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:22:55 -08:00
Jeromy
18099abb93 if bucket doesnt have enough peers, grab more elsewhere 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:22:55 -08:00
Jeromy
5e99be9899 comment multiple dagstore error checking 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:22:55 -08:00
Jeromy
efac042e82 rework editor creation and finalization 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:22:55 -08:00
Jeromy
d8ee7dffc3 improves memory usage of add 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:22:55 -08:00
Andrew Chin
b6f28dad1b Add a --pin option to ipfs add (allowing --pin=false) 
Implements a solution for #1908

This PR replaces #1909

License: MIT
Signed-off-by: Andrew Chin <achin@eminence32.net>
 2016-01-12 08:22:55 -08:00
rht
86901aff3c Move parts of ipfs add into core/coreunix 
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
 2016-01-12 08:22:55 -08:00
Jeromy
38fab91013 address comments from CR 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:22:55 -08:00
Jeromy
6faa70ee59 implement ipfs files command 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:22:55 -08:00
Jeromy
13ef17ef16 fixup comments 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:22:55 -08:00
Jeromy
78a8088410 Refactor ipnsfs into a more generic and well tested mfs 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:22:55 -08:00
Tommi Virtanen
e49e610b07 p2p/net/filter: Guard with a mutex 
Callers assume this is safe to call whenever, let's make it so.

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
 2016-01-12 08:22:55 -08:00
Tommi Virtanen
6e126ed79d fuse/ipns: Only change file size in Setattr if asked to 
This used to cause files e.g. being edited with `vi` to become 0-size.

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
 2016-01-12 08:22:55 -08:00
Tommi Virtanen
fbd9cabd93 fuse/ipns, fuse/readonly: Let the fuse library set defaults for Attr 
Without this, all entries will have nlink==0, which confuses a bunch
of tools. Most dramatically, systemd-nspawn enters a busy loop in its
lock utility function.

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
 2016-01-12 08:22:55 -08:00
Tommi Virtanen
1d21fad697 fuse/readonly: Fix importer.BuildTrickleDagFromReader call 
Last argument was dropped in ffd4c3f4db4be0c9e36c1645fd1b5a6c8e0d8b01

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
 2016-01-12 08:22:55 -08:00
Jeromy
c6ee2058d9 comments from CR 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:22:55 -08:00
Jeromy
45d4b1a8bc fixup datastore interfaces 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:22:55 -08:00
Tommi Virtanen
3ffebd942f Record datastore metrics for non-default datastores 
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
 2016-01-12 08:22:55 -08:00
Tommi Virtanen
8f2d820412 S3 datastore support 
To test it, set up an S3 bucket (in an AWS region that is not US
Standard, for read-after-write consistency), run `ipfs init`, then
edit `~/.ipfs/config` to say

      "Datastore": {
        "Type": "s3",
        "Region": "us-west-1",
        "Bucket": "mahbukkit",
        "ACL": "private"
      },

with the right values. Set `AWS_ACCESS_KEY_ID` and
`AWS_SECRET_ACCESS_KEY` in the environment and you should be able to
run `ipfs add` and `ipfs cat` and see the bucket be populated.

No automated tests exist, unfortunately. S3 is thorny to simulate.

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
 2016-01-12 08:22:55 -08:00
Tommi Virtanen
1174aab86b thirdparty/s3-datastore: Let caller set ACL, change default to safer "private" 
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
 2016-01-12 08:22:55 -08:00
Tommi Virtanen
33d41285d6 thirdparty/s3-datastore: Datastore keys can be binary, hex encode them for S3 
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
 2016-01-12 08:22:55 -08:00
Tommi Virtanen
9497e26fbd Implement pluggable Datastore types, with nothing implemented yet 
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
 2016-01-12 08:22:55 -08:00
Tommi Virtanen
b76581d6c7 fsrepo: Refactor to extract datastore internals 
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
 2016-01-12 08:22:55 -08:00
Tommi Virtanen
dc73744082 fsrepo: Detect uninitialized repo by missing config file 
Earlier, it also checked checked the leveldb directory. That part
added no crash safety to the application, and just hardcoded
assumptions about the datastore.

If anything, this should rely on the absolute last item created by
fsrepo.Init, and there should be fsync guarantees about ordering.

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
 2016-01-12 08:22:55 -08:00
Tommi Virtanen
6996ce6758 Remove Config file section "Datastore", it's not used 
This gives us a clean slate for the new code, avoiding leftovers.

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
 2016-01-12 08:22:55 -08:00
Tommi Virtanen
2c4a9ab7e4 fsrepo/serialize tests: Stop assuming internals of Config 
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
 2016-01-12 08:22:55 -08:00
Tommi Virtanen
d83644628b sharness: Stop assuming leveldb Datastore 
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
 2016-01-12 08:22:55 -08:00
Tommi Virtanen
4a33683443 core tests: Stop assuming internals of Config 
License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
 2016-01-12 08:22:55 -08:00
Tommi Virtanen
a5f9832937 gofmt generated assets 
The generated file went through some changes because of differing
go-bindata versions.

License: MIT
Signed-off-by: Tommi Virtanen <tv@eagain.net>
 2016-01-12 08:22:55 -08:00
Jeromy
e431f35a0c update multistream naming of lazyconn 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:21:13 -08:00
Jeromy
2c4eb60961 allow multistream to have zero rtt stream opening 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:21:13 -08:00
Jeromy
859de51406 dont use searchset for indirect pin checking 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:21:13 -08:00
Jeromy
879cfeeec9 pin rm fails appropriately for indirect pins 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:21:13 -08:00
Jeromy
16c4d8cdcb addressing comments from CR 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:21:13 -08:00
Jeromy
5123857c4b break up GC logic 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:21:13 -08:00
Jeromy
b12ee40aba implement mark and sweep GC 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

dont GC blocks used by pinner

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

comment GC algo

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

add lock to blockstore to prevent GC from eating wanted blocks

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

improve FetchGraph

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

separate interfaces for blockstore and GCBlockstore

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>

reintroduce indirect pinning, add enumerateChildren dag method

License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:21:13 -08:00
Juan Batiz-Benet
27f34b4311 renamed {R,}Lock -> {Pin,GC}Lock 
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
 2016-01-12 08:21:13 -08:00
Jeromy
7b675e870c lock blockstore for pin add 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:21:13 -08:00
Jeromy
1a03d5e37a move locking out of GC branch 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:21:13 -08:00
Jeromy
dc3b9ed140 address concerns from PR 
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
 2016-01-12 08:21:13 -08:00