Integrate the existing v2 APIs into Lotus Gateway such that they are
accessible via `/rpc/v2` endpoint similar to full node APIs except
through the gateway reverse proxy along with rate limiting and handling
of stateful connections over web socket.
The prior gateway implementation did not have a concept of per-uri
reverse proxy handling. The changes here separate reverse proxy
implementation in gateway from the gateway node to introduce separate
handling of v1 and v2 APIs. The work introduces a top-level
`v2api.Gateway` interface to capture the APIs exposed by the v2 gateway
and generate any relevant documentation based on the interface.
Specifically, the work here separates the handling of stateful
connections between v1 and v2 to avoid any cross-contamination between
Eth calls to either API. However, the total rate limiting has stayed
shared across both URIs.
The underused mock interface in v1 is now removed in favor of the
generated gomock interfaces to reduce the LOC in `gateway` package. The
interface was barely used by the tests and would have resulted in
unnecessary LOC to implement v2.
I have held back on unrelated refactoring of `gateway` package. The code
can use further restructuring, but for now the focus is to expose
existing /v2 APIs over the gateway.
The implementation of `Version` in /v2 APIs is also left as a TODO since
it requires implementation at the top level API and bubbles up into
`Common` APIs.
Introduce the first API for Lotus v2, focusing on `ChainGetTipSet`
within the `Chain` group. Define `TipSetSelector` for advanced
tipset retrieval options, and create a compact JSON-RPC call format.
Gracefully accommodate both EC and F3 finalized tipsets based on node
configuration, where:
* EC finalized tipset is returned when F3 is turned off, has no
finalized tipset or F3 isn't ready.
* F3 finalized is returned otherwise.
Support three categories of selectors under `TipSetSelector`:
* By tag: either "latest" or "finalized."
* By height: epoch, plus optional fallback to previous non-null tipset.
* By tipset key.
The selection falls back to tag "latest" if the user specifies no
selection criterion.
The JSON-RPC format is designed to use JSON Object as the parameters
passed to the RPC call to remain compact, and extensible.
Upgrade to go-f3 `v0.8.3`
Upgrade to the latest go-f3, to bubble up:
* various bug fixes
* extended manifest parameters for better control in passive testing
Note that the return type from F3GetProgress API has changed to now
include the input chain to an instance.
Update CLI templates to print the additional fields.
Upgrade to the latest go-f3 and allow F3 chain exchange topics
Upgrade to the latest `go-f3` and add the F3 chain exchange topics to
the allowed topic list for both static and dynamic manifests.
Fixes https://github.com/filecoin-project/go-f3/issues/809
Automatically detect if genesis CAR is compressed
When `--genesis` path is set, automatically detect if the genesis file
is ZSTD compressed and decompress it.
Log failure to run CLI command
Prior code only conditionally logged the output when `LOTUS_DEV` env
var was present. This doesn't make sense for a number of reasons:
* In case of an error starting up `lotus daemon` via systemctl for
example the logs would miss the error. One then has to search through
system journal to see what happened.
* There are no other places that I can find where `LOTUS_DEV` env var is
used. The commit that introduced this condition is very old with no
clear commit message to shed light into the rationale.
For the first reason alone, the changes here remove that condition, and
log the message when the logging output is not standard err or out. This
way we at least cover the case where lotus is run in a production
environment and would avoid partial logging of errors.
* Switch to pure-go zstd decoder for snapshot imports
The benchmarking performed as part of #12852 shows that the pure-go
implementation of zstd decoder is fast-enough for snapshot import.
Switch to it motivated by the desire to reduce CGO dependencies across
Lotus.
Fixes#12852
* Update the pain-log.
Handle non-existing actors gracefully in F3 power proportion CLI
When calculating proportional F3 participation power for a given actor
IDs instead of failing when an actor isn't found, collect the
non-existing ones and report them. This makes up a better UX when
debugging F3 in cases where actors don't exist in the F3 power table.
* docs: Update format flag usage with valid placeholders for ChainListCmd
Signed-off-by: Jakub Sztandera <oss@kubuxu.com>
* feat: Add <json_tipset> placeholder for JSON encoding of the tipset
Signed-off-by: Jakub Sztandera <oss@kubuxu.com>
* style: Fix formatting in chain.go for consistency and readability
Signed-off-by: Jakub Sztandera <oss@kubuxu.com>
* fix: Change JSON encoding to single line without indentation
Signed-off-by: Jakub Sztandera <oss@kubuxu.com>
* docs: Update Usage for format flag to remove placeholder explanations
Signed-off-by: Jakub Sztandera <oss@kubuxu.com>
* feat: Rename 'height' flag to 'epoch' in ChainListCmd and update usage
Signed-off-by: Jakub Sztandera <oss@kubuxu.com>
* add changelog
Signed-off-by: Jakub Sztandera <oss@kubuxu.com>
* docs: Update CLI options in lotus documentation for clarity and detail
Signed-off-by: Jakub Sztandera <oss@kubuxu.com>
* fix changelog
Signed-off-by: Jakub Sztandera <oss@kubuxu.com>
---------
Signed-off-by: Jakub Sztandera <oss@kubuxu.com>
* Implement F3 CLI to list power table and proportional power at instance
Implement utility CLIs to:
* get the power table used by F3 at a given instance ID.
* get total proportional power of a list of actors at a given instance
ID.
These utilities allow us to debug the exact participation power for an
instance without having to manually calculate it or estimate it from
the latest power.
* Update changelog
* Address lint issue
* Regenerate CLI docs
* Take instance ID via flag and actor IDs as args
* Reduce indentation by defiling top level vars
* Work around bug in docsgencli by using one-liner usage
* When no range is given, default limit to 10
* Otherwise default to unlimited.
This is better than the current default of always unlimited because:
* there are a lot of certs, and
* when range is given the chances are the user wants all of them.
Addresses https://github.com/filecoin-project/lotus/pull/12627#discussion_r1815426539
* fix(api): make stick sessions actually work and make them non-racy
We apparently have a way to specify that all "related" requests should
go to the same node. However:
1. It didn't work at all. All future requests would go to the first successful
node from the first request. Because that's how stack variables work.
2. It was racy if the context was re-used concurrently. But only the
first time, see point 1.
* test(api): test the API merge proxy
1. Test whether or not it works.
2. Test stickiness.
* fix(api): update OnSingleNode documentation
Implement `lotus f3` CLI sub commands to:
* Get a specific finality certificate, either latest or by instance ID.
* List a range of finality certificates
Part of #12607
Implement `lotus f3` CLI sub commands:
* `manifest` to dump the current F3 manifest in either JSON or text.
* `list-miners` to list the current F3 participants.
* `status` to print summary status of F3.
Part of #12607
* feat: add settle deal command for lotus miner [WIP]
* feat: allow settlement of all deals for the miner and fix ci jobs
* fix: address comments
* fix: address comment
