This new file comes from:
https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-test.txt
It is licensed under CC BY 4.0.
After discussing that with its author, I think it is ok to
add it as is to our repository. The only restriction might
be that we should indicate in the file any change we make
to it.
License: CC BY 4.0
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
IPNSHostnameOption() touches the URL path only on the way in,
but not on the way out. This commit makes it complete by
touching the following URLs in responses:
- Heading, file links, back links in directory listings
- Redirecting /foo to /foo/ if there's an index.html link
- Omit Suborigin header
License: MIT
Signed-off-by: Lars Gierth <larsg@systemli.org>
Also, now, if ipfs config foo.bar has value of anything that is not map (0, "0", 0.1),
then ipfs config foo.bar.baz now returns an error instead of a panic
License: MIT
Signed-off-by: rht <rhtbot@gmail.com>
This changes the pin behavior. It uses the filenames given through
the api, and allows files to be streamed faltly (not a hierarchy),
which is easier for other things (like vinyl in node-ipfs-api land).
Files can also be entirely out of order, and the garbage intermediate
directories will not be pinned (gc-ed later).
The changes also mean the output of add has changed slightly-- it
no longer shows the local path added, but rather the dag path
relative to the added roots. This is a small difference, but changes
tests.
The dagutils.Editor creates a lot of chaff (intermediate objects)
along the way. Wonder how we might minimize the writes to the
datastore...
This commit also removes the "NilRepo()" part of the --only-hash
mode. We need to store at least in an in-mem repo/datastore because
otherwise the dagutils.Editor breaks.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
implement rabin fingerprinting as a chunker for ipfs
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
vendor correctly
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
refactor chunking interface a little
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
work chunking interface changes up into importer
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
move chunker type parsing into its own file in chunk
License: MIT
Signed-off-by: Jeromy <jeromyj@gmail.com>
up until now there has been a very annoying bug with get, we would
get halting behavior. I'm not 100% sure this commit fixes it,
but it should. It certainly fixes others found in the process of
digging into the get / tar extractor code. (wish we could repro
the bug reliably enough to make a test case).
This is a much cleaner tar writer. the ad-hoc, error-prone synch
for the tar reader is gone (with i believe was incorrect). it is
replaced with a simple pipe and bufio. The tar logic is now in
tar.Writer, which writes unixfs dag nodes into a tar archive (no
need for synch here). And get's reader is constructed with DagArchive
which sets up the pipe + bufio.
NOTE: this commit also changes this behavior of `get`:
When retrieving a single file, if the file exists, get would fail.
this emulated the behavior of wget by default, which (without opts)
does not overwrite if the file is there. This change makes get
fail if the file is available locally. This seems more intuitive to
me as expected from a unix tool-- though perhaps it should be
discussed more before adopting.
Everything seems to work fine, and i have not been able to reproduce
the get halt bug.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
this PR greatly speeds up providing and add.
(1) Instead of idling workers, we move to a ratelimiter-based worker.
We put this max at 512, so that means _up to_ 512 goroutines. This
is very small load on the node, as each worker is providing to the
dht, which means mostly waiting. It DOES put a large load on the DHT.
but i want to try this out for a while and see if it's a problem.
We can decide later if it is a problem for the network (nothing
stops anyone from re-compiling, but the defaults of course matter).
(2) We add a buffer size for provideKeys, which means that we block
the add process much less. this is a very cheap buffer, as it only
stores keys (it may be even cheaper with a lock + ring buffer
instead of a channel...). This makes add blazing fast-- it was being
rate limited by providing. Add should not be ratelimited by providing
(much, if any) as the user wants to just store the stuff in the local
node's repo. This buffer is initially set to 4096, which means:
4096 * keysize (~258 bytes + go overhead) ~ 1-1.5MB
this buffer only last a few sec to mins, and is an ok thing to do
for the sake of very fast adds. (this could be a configurable
paramter, certainly for low-mem footprint use cases). At the moment
this is not much, compared to block sizes.
(3) We make the providing EventBegin() + Done(), so that we can
track how long a provide takes, and we can remove workers as they
finish in bsdash and similar tools.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
Fedora/RedHat distros comply with US patent law and remove this curve,
which makes it impossible to run ipfs with distro provided Golang.
License: MIT
Signed-off-by: Pavol Rusnak <stick@gk2.sk>
This patch update fixes a problem we introduced in 0.3.6 and did not
catch: the webui failed to work with out-of-the-box CORS configs.
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
need to do it this way to avoid VERY confusing situations where
the user would change the API port (to another port, or maybe even
to :0). this way things dont break on the user, and by default,
users only need to change the API address and things should still
"just work"
License: MIT
Signed-off-by: Juan Batiz-Benet <juan@benet.ai>
