Since go1.16, there are built in tools that allow for embeding
filesystem inside the binary. We now make use of the `embed` package to
have all files put into the binary, removing the need to generate the
files and removes dependencies
Co-authored-by: Jorropo <jorropo.pgm@gmail.com>
* fix: report gateway http metrics only when response is successful
* fix(gw): 304 Not Modified as no-op
This fix ensures we don't do any additional work when Etag match
what user already has in their own cache.
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* add deprecation warning when tracer plugins are loaded
* add response format attribute to span in gateway handler
* add note about tracing's experimental status in godoc
* add nil check for TTL when adding name span attrs
* add basic sharness test for integration with otel collector
* add nil check in UnixFSAPI.processLink
* test: sharness check all json objs for swarm span
* add env var docs to docs/environment-variables.md
* chore: pin the otel collector version
* add tracing spans per response type (#8841)
* docs: tracing with jaeger-ui
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* fix(gw): validate requested CAR version
This adds validation of 'application/vnd.ipld.car;version=n' passed
in the Accept header by HTTP clients to align Gateway behavior with
the spec submitted to IANA.
* test: fix comment in test/sharness/t0118-gateway-car.sh
Co-authored-by: Gus Eggert <gus@gus.dev>
Co-authored-by: Gus Eggert <gus@gus.dev>
* feat(gw): response type histogram metrics
- response-type agnostic firstContentBlockGetMetric which counts the
latency til the first content block.
- car/block/file/gen-dir-index duration histogram metrics that show how
long each response type takes
* docs: improve metrics descriptions
* feat: more gw histogram buckets
0.05, 0.1, 0.25, 0.5, 1, 2, 5, 10, 30, 60 secs
as suggested in reviews at https://github.com/ipfs/go-ipfs/pull/8443
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Gus Eggert <gus@gus.dev>
* feat: serveRawBlock implements ?format=block
* feat: serveCar implements ?format=car
* feat(gw): ?format= or Accept HTTP header
- extracted file-like content type responses to separate .go files
- Accept HTTP header with support for application/vnd.ipld.* types
* fix: use .bin for raw block content-disposition
.raw may be handled by something, depending on OS, and .bin
seems to be universally "binary file" across all systems:
https://en.wikipedia.org/wiki/List_of_filename_extensions_(A%E2%80%93E)
* refactor: gateway_handler_unixfs.go
- Moved UnixFS response handling to gateway_handler_unixfs*.go files.
- Removed support for X-Ipfs-Gateway-Prefix (Closes#7702)
* refactor: prefix cleanup and readable paths
- removed dead code after X-Ipfs-Gateway-Prefix is gone
(https://github.com/ipfs/go-ipfs/issues/7702)
- escaped special characters in content paths returned with http.Error
making them both safer and easier to reason about (e.g. when invisible
whitespace Unicode is used)
* fix: add companion ids to allow origins
- fixes#8689
- Adds the chrome-extension ids for ipfs-companion and
ipfs-companion-beta to the allowed origins list, this
allows us to accesss ipfs api from a manifest v3 extension.
- added tests in t0401-api-browser-security.sh
* fix: companion when custom CORS *-Origin is set
Companion extension should be able to access RPC API even when custom
Access-Control-Allow-Origin is set
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* chore(gateway): better logging for the http requests
* chore(gateway): removed defer and add more data to the final log
* chore(gateway): debug logging refactor
* chore(gateway): use debug w/o context when only msg
* doc: add cmd for log level
* chore: add more logs and address fedback
* chore(gateway): log subdomains and from=requestURI, refactor
* chore(gateway): fix debug redirect
* plumb through go-datastore context changes
* update go-libp2p to v0.16.0
* use LIBP2P_TCP_REUSEPORT instead of IPFS_REUSEPORT
* use relay config
* making deprecation notice match the go-ipfs-config key
* docs(config): circuit relay v2
* docs(config): fix links and headers
* feat(config): Internal.Libp2pForceReachability
This switches to config that supports setting and reading
Internal.Libp2pForceReachability OptionalString flag
* use configuration option for static relays
* chore: go-ipfs-config v0.18.0
https://github.com/ipfs/go-ipfs-config/releases/tag/v0.18.0
* feat: circuit v1 migration prompt when Swarm.EnableRelayHop is set (#8559)
* exit when Swarm.EnableRelayHop is set
* docs: Experimental.ShardingEnabled migration
This ensures existing users of global sharding experiment get notified
that the flag no longer works + that autosharding happens automatically.
For people who NEED to keep the old behavior (eg. have no time to
migrate today) there is a note about restoring it with
`UnixFSShardingSizeThreshold`.
* chore: add dag-jose code to the cid command output
* add support for setting automatic unixfs sharding threshold from the config
* test: have tests use low cutoff for sharding to mimic old behavior
* test: change error message to match the current error
* test: Add automatic sharding/unsharding tests (#8547)
* test: refactored naming in the sharding sharness tests to make more sense
* ci: set interop test executor to convenience image for Go1.16 + Node
* ci: use interop master
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Marten Seemann <martenseemann@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Co-authored-by: Gus Eggert <gus@gus.dev>
Co-authored-by: Lucas Molas <schomatis@gmail.com>
Cannot reproduce the flakiness at the moment. The report suggests that connections are established on different transports. Adding logging to show what these transports are.
I believe we figured that these were for "informational purposes", but
really, we _should_ always be able to resolve names to CIDs. If we
can't, there's probably something wrong with the directory.
Namesys is a very useful submodule. Given a ValueStore and a Datastore it can
resolve and publish /ipns/ paths.
This functionality does not need to be sequestered inside go-ipfs as it can
and should be used without IPFS, for example, for implementing lightweight
IPNS publishing services or for resolving /ipns/ paths.
"keystore" extraction was necessary, as there is a dependency to it in
namesys. Keystore is also a useful module by itself within the stack.
Fixes#6537
We used Clear-Site-Data to cushion transition period for local gateway
exposed at http://localhost while we were still figuring out
security-related details.
In the final implementation subdomain gateways are not tied to a
hostname explicitly, which removes the risk of cookies leaking,
removing the need for the header.
Turns out it causes issues for Firefox users, so let's just remove it.
Closes https://github.com/ipfs-shipyard/ipfs-companion/issues/977
