98 Commits

Author SHA1 Message Date
48cd35195a fix: allow users to optionally configure node startup even if index reconciliation fails (#12930)
* optional index start on reconciliation failure

* warning

* fix ChangeLog

* fix make gen

* fix compilation
2025-03-06 11:43:38 +04:00
96684f7c68 Add StateCall to StateModuleAPI, it is already provided by Gateway
Signed-off-by: Jakub Sztandera <oss@kubuxu.com>
2025-02-19 18:49:01 +01:00
f3e0cf7a42 Fix the builder
Signed-off-by: Jakub Sztandera <oss@kubuxu.com>
2025-02-19 18:49:00 +01:00
be8e782853 WIP: f3-activation-contract integration
Includes shed tool for debuging, the tool could be re-written to use the
the ContractManifestProvider.

Integration of ContractManifestProvider into the existing manifest
pipeline is missing.

Signed-off-by: Jakub Sztandera <oss@kubuxu.com>
2025-02-19 18:49:00 +01:00
8972194753 fix: Only run F3 on full nodes
It has no chance of working on lite nodes right now

Signed-off-by: Jakub Sztandera <oss@kubuxu.com>
2025-02-19 18:48:54 +01:00
edfa817daf chore(eth): refactor eth API module into separate pieces in new pkg (#12796) 2024-12-20 08:32:46 +00:00
9bfd0b4295 chore: remove IPNI advertisement relay over pubsub via Lotus node (#12768)
The initial implementation of IPNI used GossipSub as a way to propagate
IPNI advertisement chain. To do this the propagation had to be relayed
through the Lotus node due to strict Filecoin GossipSub validation
rules.

Since then IPNI has moved on to roll out its own sync protocol that
works over HTTP, and HTTP-over-libp2p. This move has been the official
way of advertising content to IPNI federation over a year now.

Therefore, remove the ad relay over pubsub via Lotus node as it is now
considered to have reached its EOL as a mechanism for advertising to
IPNI.
2024-12-10 12:56:04 +00:00
1f5f51fc38 doc(indexer): more clarity for ChainIndexer configuration setup (#12701) 2024-11-19 07:51:35 +00:00
dcc903c65d feat: a new ChainIndexer to index tipsets, messages and events (#12421)
* chain index complete for msgs and txns

* dont need observer changes for now

* changes

* fix tests

* fix tests

* use th right context

* index empty tipsets correctly

* implement automated backfilling

* add event indexing and remove all old indices

* fix test

* revert deployment test changes

* revert test changes and better error handling for eth tx index lookups

* fix sql statments naming convention

* address review for Index GC

* more changes as per review

* changes as per review

* fix config

* mark events as reverted during reconciliation

* better reconciliation; pens down and code complete; also reconcile events

* fix tests

* improve config and docs

* improve docs and error handling

* improve read logic

* improve docs

* better logging and handle ennable event storage

* improve logs and index init proc

* better logging

* fix bugs based on calibnet testing

* create sqliite Indices

* gc should be based on epochs

* fix event query

* foreign keys should be enabled on the DB

* reverted tipsets should be removed as part of GC

* release read lock

* make it easy to backfill an empty index using reconciliation

* better docs for reconciliation

* fix conflicts with master

* Apply suggestions from code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* fix go mod

* fix formatting

* revert config changes

* address changes in observer

* remove top level chainindex package

* changes as per review

* changes as per review

* changes as per review

* handle index with reverted tipsets during reconciliation

* changes as per review

* fix type of max reconcile epoch

* changes to reconciliation as per review

* log ipld error

* better logging of progress

* disable chain indexer hydrate from snapshot based on config

* always populate index

* make config easy to reason about

* fix config

* fix messaging

* revert config changes

* Apply suggestions from code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* changes as per review

* make error messages homogenous

* fix indentation

* changes as per review

* feat: recompute tipset to generate missing events if event indexing is enabled (#12463)

* auto repair events

* make jen

* fix leaky abstraction

* better docs for gc retention epoch

* imrpove DB handling (#12485)

* fix conflict

* fix lite node config for indexer

* exclude reverted events from eth get logs if client queries by epoch

* Simply addressing for event lookups in the index.

simply addressing for event lookups

* Apply suggestions from code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* Apply suggestions from code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* fix tests

* Apply suggestions from code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* feat: migration("re-indexing"), backfilling and diasgnostics tooling for the `ChainIndexer` (#12450)

* fix conflicts with chain indexer

* feat: chain indexer todos [skip changelog] (#12462)

* feat: finish todos of validation api

* feat: add indexed data verification with chain store

* feat: address comments and finish TODO

* fix: build issue

* address comments

* fix: ci issue

* Apply suggestions from code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* changes to Index Validation API based on Rodds first review

* build chain indexer API

* improve error handling

* feat: lotus-shed tooling for chain indexer (#12474)

* feat: add lotus-shed command for backfilling chain indexer

* feat: add lotus-shed command for inspecting the chain indexer

* feat: use single lotus-shed command to inspect and backfill

* fix: remove the unused queries

* small changes

* add change log

* backfilling improvements and fixes

* finish chain index validation and backfill tooling

* user documentation for the

* validate from epoch

* Apply suggestions from code review

Suggestions from Steve's read of the user doc.

Co-authored-by: Steve Loeppky <biglep@filoz.org>

* changes to user doc as per review

* Apply suggestions from code review

Co-authored-by: Steve Loeppky <biglep@filoz.org>

* changes to user doc as per review

* Apply suggestions from code review

Co-authored-by: Steve Loeppky <biglep@filoz.org>

* changes as per review

* feat: add event entries count in validation API (#12506)

* feat: add event entry count in validation API

* address comments

* use sqllite defaults (#12504)

* Apply suggestions from code review

Co-authored-by: Steve Loeppky <biglep@filoz.org>

* write chain index to a different dir

* Apply suggestions from code review

Co-authored-by: Steve Loeppky <biglep@filoz.org>

* fix conflicts

* UX improvements to backfilling

* feat: tests for the chain indexer (#12521)

* ddl tests

* tests for the chain indexer

* finish unit tests for chain indexer

* fix formatting

* cleanup reverted tipsets to avoid db bloat

* fix logging

* test for filter by address

* test gc cascade delete

* fix db locked error during backfilling

* fix var name

* increase db locked timeout

* fix db locked issue

* reduce db lock timeout

* no lock in gc

* reconcile does not need lock

* improved error handling

* Update chain-indexing-overview-for-rpc-providers.md

Doc updates based on @jennijuju feedack.

* Update chain-indexing-overview-for-rpc-providers.MD

Fixes after reviewing 33c1ca1831

* better metrics for backfilling

* Update chain/index/chain-indexing-overview-for-rpc-providers.MD

Co-authored-by: Rod Vagg <rod@vagg.org>

* Update chain/index/chain-indexing-overview-for-rpc-providers.MD

Co-authored-by: Rod Vagg <rod@vagg.org>

* Update chain/index/chain-indexing-overview-for-rpc-providers.MD

Co-authored-by: Rod Vagg <rod@vagg.org>

* Update chain/index/chain-indexing-overview-for-rpc-providers.MD

Co-authored-by: Rod Vagg <rod@vagg.org>

* Update chain/index/chain-indexing-overview-for-rpc-providers.MD

Co-authored-by: Rod Vagg <rod@vagg.org>

* Update chain/index/chain-indexing-overview-for-rpc-providers.MD

Co-authored-by: Rod Vagg <rod@vagg.org>

* Update chain/index/chain-indexing-overview-for-rpc-providers.MD

Co-authored-by: Rod Vagg <rod@vagg.org>

* tests for changes to event addressing

* Apply suggestions from code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* changes as per review -> round 1

* Apply suggestions from code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* Apply suggestions from code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* log tipset key cid

* Apply suggestions from code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* Apply suggestions from code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* fix docs

* Apply suggestions from code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* fix tests

* fix tests

* make jen

* fix conflicts

---------

Co-authored-by: Aryan Tikarya <aryan.tikarya@dojima.network>
Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Steve Loeppky <biglep@filoz.org>

* fix lint

* Apply suggestions from code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* Apply suggestions from code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* remove reverted flag from RPC

* Apply suggestions from code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* fix testing of events and dummy chain store

* remove lotus shed commands for old Indices

* change type of event counts to uint64

* only recompute events if theyre not found

* short-circuit empty events path for older tipsets

* chain indexer must be enabled if ETH RPC is enabled

* change name of message_id column to id in tipset_message table

* only expose SetRecomputeTipSetStateFunc

* dont block on head indexing for reading messages

* document why we're only checking for missing events for a single tipset

* document when we query for reverted events

* simplify event collection

* Apply suggestions from code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* fix test

* change event_id to id in the event table

* change head indexed timeout

* remove deprecated config options

* fail ETH RPC calls if ChainIndexer is disabled

* fix docs

* remove the tipset key cid func from lotus shed

* address review comments

* Apply suggestions from code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* chore(events): remove unnecessary DisableRealTimeFilterAPI (#12610)

* feat(cli): add --quiet to chainindex validate-backfill + cleanups (#12611)

* fix tests

* Apply suggestions from code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* error type for disabled chainindexer

* fix(chainindex): recompute tipset when we find no receipts

* fix(chainindexer): backfilling should halt when chain state data is missing and not backfill parents (#12619)

* fix backfilling UX

* Update chain/index/api.go

Co-authored-by: Rod Vagg <rod@vagg.org>

* address review

---------

Co-authored-by: Rod Vagg <rod@vagg.org>

* reduce log noise

* make jen

* make jen

* docs: finishing chain-indexer-overview-for-operators.md (#12600)

* Followup to PR #12450 for doc updates

This is being used to resolve the unresolved items in https://github.com/filecoin-project/lotus/pull/12450 since that PR is unwieldly at this point.

* Incorporated some items and added TODOs based on unresolved items from https://github.com/filecoin-project/lotus/pull/12450

* Incorporating more feedback

* Pointing to issue to learn about benefits

* Formatting fixes

* Apply most of the suggestions from @rvagg code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* Incorporating feedback from https://github.com/filecoin-project/lotus/pull/12600#discussion_r1802519453

* Addressing https://github.com/filecoin-project/lotus/pull/12600#discussion_r1802540042 and more

* Moved chain-indexer docs to documentation
Renamed
Added ToC

We can move to lotus-docs later

* Update documentation/en/chain-indexer-overview-for-operators.md

Co-authored-by: Rod Vagg <rod@vagg.org>

* Update documentation/en/chain-indexer-overview-for-operators.md

Co-authored-by: Rod Vagg <rod@vagg.org>

* Added upgrade path when importing chain state from a snapshot.

* Typo fixes

* Update documentation/en/chain-indexer-overview-for-operators.md

Co-authored-by: Rod Vagg <rod@vagg.org>

* chore(doc): "regular checks" section for chainindexer docs (#12612)

* Apply suggestions from @rvagg code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* Incorporating @aarshkshah1992 feedback

* Update documentation/en/chain-indexer-overview-for-operators.md

Co-authored-by: Rod Vagg <rod@vagg.org>

---------

Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>

* remove go mod replace

* remove unnecessary changes from CHANGELOG

* fix test

* compare events AMT root (#12632)

* fix(chainindex): retry transaction if database connection is lost (#12657)

* retry database lost connection

* log context cancellation

* address review

* fix gateway itest: no chainindexer for lite nodes

* fix changelog

---------

Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Aryan Tikarya <aryan.tikarya@dojima.network>
Co-authored-by: Steve Loeppky <biglep@filoz.org>
2024-10-31 09:58:19 +00:00
1b9b81537b feat(f3): prepare for f3 bootstrap (#12552)
This patch:

1. Bootstraps F3 when we hit the `F3BootstrapEpoch` (when non-negative).
2. Refuses any/all dynamic manifests once we get within one finality of said epoch.
3. Sets the F3 network name for mainnet to "filecoin".
4. Refuses any/all dynamic manifests that don't start with the expected
    network name prefix.
2024-10-07 15:03:55 +00:00
0e7292acc7 chore(f3): update go-f3 to 0.4.0 (#12547)
---

Signed-off-by: Jakub Sztandera <oss@kubuxu.com>
Co-authored-by: Steven Allen <steven@stebalien.com>
2024-10-03 18:50:13 +00:00
841041e0b8 feat(f3): add a build parameter to set the initial power table cid (#12502)
This will be used post-bootstrap to hard-code the initial F3 power table
CID.

If and only if it has not already hard-coded, it can be specified by setting the
F3_INITIAL_POWERTABLE_CID environment variable.
2024-09-24 13:37:39 +00:00
a30d024323 test(f3): f3 integration tests (#12486)
Co-authored-by: Alfonso de la Rocha <adlrocha@tutamail.com>
Co-authored-by: Jakub Sztandera <oss@kubuxu.com>
2024-09-20 16:22:03 +00:00
d8d699a2b0 feat(f3): checkpoint tipsets that are finalized by F3 (#12460)
* Checkpoint tipsets that are finalized by F3

Once a decision is received from F3, checkpoint it in chain-store.

As part of checkpointing, refactor the F3 tipset wrapper to reduce type
casting.

Note that go-f3 module now uses Go 1.22, which requires Lotus go module
to be updated accordingly.

Part of https://github.com/filecoin-project/go-f3/issues/603

* Make checkpointing chainstore optional via build constraints
2024-09-18 16:39:49 +01:00
42e6e41efe feat(eth): add EthSendRawTransactionUntrusted, use via gateway eth_sendRawTransaction (#12431)
* test(mpool): more complete testing of MpoolPush* methods
* feat(eth): add EthSendRawTransactionUntrusted, use via gateway eth_sendRawTransaction
* fix(eth): move EthSendRawTransactionUntrusted off main EthModuleAPI
  - Move it to EthAPI so the GatewayAPI doesn't have to implement it.
2024-09-07 08:14:12 +00:00
d7b218e148 chore: ffi: copy verifier iface, mock & ffi out of storage (#11581)
* chore: copy verifier iface, mock & ffi out of storage

one step toward a cleaner separation of miner & chain

* ffi: s/verifier/proofs, incorporate GenerateUnsealedCID

* fix: ffi: use non-ffi version of GenerateUnsealedCID

* doc: proofs: document GenerateUnsealedCID use
2024-08-09 11:04:36 +10:00
0bd11a91c6 fix: events: don't initialise events helpers when not needed
filter.EventFilterManager starts a chain observer which isn't needed if it's
not going to be consumed by events subsystems.
2024-07-30 10:30:02 +10:00
4f63a08605 fix: Eth Event Receipt Logs: use event index for logs (#12269)
* use event index for receipt logs

* tests

* increment event count if there is an address mismatch

* itests for receipt logs consistency

* update ChangeLog

* changes as per review

* do not increment event count if address resolution fails

* Apply suggestions from code review

Co-authored-by: Rod Vagg <rod@vagg.org>

* address review

---------

Co-authored-by: Rod Vagg <rod@vagg.org>
2024-07-23 13:54:55 +04:00
11dd4c9df7 chore: config: yet more lp2p removal from miner 2024-07-19 12:19:09 +04:00
44569437a0 feat:ec: integrate F3 dynamic manifest (#12173)
* F3-370: integrate F3 dynamic manifest

* F3-370: make linter happy

* Set manifest sender identities

* Decode manifest sender peer ID from string before using it

Peer ID is of type string internally but the internal string
representation is not the same as the encoded string representation.
Therefore, the latter needs to be decoded and cannot be casted to the
former. Otherwise, it will represent a different ID.

* `make gen` the pain of my life

---------

Co-authored-by: Masih H. Derkani <m@derkani.org>
2024-07-04 16:04:46 -04:00
7e37ee1b88 feat:ec:f3: add env var to disable F3 (#12176)
* F3-388: add env var to disable F3

Signed-off-by: Jakub Sztandera <oss@kubuxu.com>

* F3-388: check env variable in pubsub also

Signed-off-by: Jakub Sztandera <oss@kubuxu.com>

* F3-388: use DISABLE_F3=1 instead of _yes_ like in drand

Signed-off-by: Jakub Sztandera <oss@kubuxu.com>

* Use IsF3Enabled() and remove the runtime stopping

Signed-off-by: Jakub Sztandera <oss@kubuxu.com>

---------

Signed-off-by: Jakub Sztandera <oss@kubuxu.com>
Co-authored-by: Jakub Sztandera <oss@kubuxu.com>
2024-07-04 17:48:50 +02:00
7cdf9aa2cd feat: ec: Integrate Fast Finality for Filecoin (#12119)
Integrate F3 into Lotus

Signed-off-by: Jakub Sztandera <oss@kubuxu.com>
Co-authored-by: Jennifer Wang <jiayingw703@gmail.com>
Co-authored-by: Masih H. Derkani <m@derkani.org>
Co-authored-by: Steven Allen <steven@stebalien.com>
2024-06-28 17:10:58 +00:00
e3deda0b2b cleanup: miner: remove markets and deal-making from Lotus Miner (#12005)
* remove client CLI

* remove markets CLI from miner

* remove markets from all CLI

* remove client API

* update go mod

* remove EnableMarkets flag

* remove market subsystem

* remove dagstore

* remove index provider

* remove graphsync and data-transfer

* remove markets

* go mod tidy

* fix cbor gen deps

* remove deal making from config

* remove eol alert

* go mod tidy

* changes as per review

* make jen

* changes as per review

* merge master

* remove libp2p from config

* miner does not have libp2p conn in api test
2024-06-05 18:14:50 +04:00
0a51a0a219 feat!: build: separate miner and node version strings
Ref: https://github.com/filecoin-project/lotus/issues/12010
2024-05-30 10:02:43 +10:00
7bd21bc675 feat: ipfs: remove IPFS client backend (#11661)
This was used to run an internal IPFS node...
2024-04-04 07:35:54 -07:00
78d9d9baa0 Merge branch 'releases' 2024-03-27 08:55:17 -07:00
1a21b42fd7 Chore: Backports to the release/v1.26.0 branch (#11713)
* enable storing events (#11712)

* fix: commit batch: Always go through commit batcher (#11704)

* fix: commit batch: Always go through commit batcher

* fix sealing fsm tests

* sealing pipeline: Fix panic on padding pieces in WaitDeals (#11708)

* sealing pipeline: Fix panic on padding pieces in WaitDeals

* sealing pipeline: Catch panics

* sealing pipeline: Output DDO pieces in SectorStatus (#11709)

* sealing pipeline: Fix failing ProveCommit3 aggregate (#11710)

* itests: Repro failing ProveCommit3 aggregate

* commit batch: Correctly sort sectors in processBatchV2

* fix imports

* ci: Bigger instance for sector_pledge test

* itests: Use Must-Post mining in TestPledgeBatching

---------

Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
2024-03-22 07:00:28 +01:00
d35ebe5bc0 feat(config): move Fevm.Events->Events, implement soft deprecation
Closes: https://github.com/filecoin-project/lotus/issues/11679

* Introduce a `moved:"To.New.Config"` tag which prints a stderr warning when
  you use one of these, but will move any set value to the new location if the
	new location isn't already set itself.
* Look for `X is DEPRECATED` to hold certain fields back from documentation.
* Use `toml:"omitempty"` to prevent the default config output from having these
  deprecated values.
2024-03-22 07:00:28 +01:00
590ce97edf s/ActorEvents/Events/g in global config 2024-03-22 07:00:28 +01:00
13d7411bd9 Adjust actor event API after review 2024-03-22 07:00:28 +01:00
4a0e3b877c Rename internal events modules for clarity 2024-03-22 07:00:28 +01:00
43ed348635 Built-in actor events first draft 2024-03-22 07:00:28 +01:00
1c1b10c66e enable storing events (#11712) 2024-03-12 18:22:02 +04:00
6f7498b622 chore: Merge nv22 into master (#11699)
* [WIP] feat: Add nv22 skeleton

Addition of Network Version 22 skeleton

* update FFI

* feat: drand: refactor round verification

* feat: sealing: Support nv22 DDO features in the sealing pipeline (#11226)

* Initial work supporting DDO pieces in lotus-miner

* sealing: Update pipeline input to operate on UniversalPiece

* sealing: Update pipeline checks/sealing states to operate on UniversalPiece

* sealing: Make pipeline build with UniversalPiece

* move PieceDealInfo out of api

* make gen

* make sealing pipeline unit tests pass

* fix itest ensemble build

* don't panic in SectorsStatus with deals

* stop linter from complaining about checkPieces

* fix sector import tests

* mod tidy

* sealing: Add logic for (pre)committing DDO sectors

* sealing: state-types with method defs

* DDO non-snap pipeline works(?), DDO Itests

* DDO support in snapdeals pipeline

* make gen

* update actor bundles

* update the gst market fix

* fix: chain: use PreCommitSectorsBatch2 when setting up genesis

* some bug fixes

* integration working changes

* update actor bundles

* Make TestOnboardRawPieceSnap pass

* Appease the linter

* Make deadlines test pass with v12 actors

* Update go-state-types, abstract market DealState

* make gen

* mod tidy, lint fixes

* Fix some more tests

* Bump version in master

Bump version in master

* Make gen

Make gen

* fix sender

* fix: lotus-provider: Fix winning PoSt

* fix: sql Scan cannot write to an object

* Actually show miner-addrs in info-log

Actually show miner-addrs in lotus-provider info-log

* [WIP] feat: Add nv22 skeleton

Addition of Network Version 22 skeleton

* update FFI

* ddo is now nv22

* make gen

* temp actor bundle with ddo

* use working go-state-types

* gst with v13 market migration

* update bundle, builtin.MethodsMiner.ProveCommitSectors2 -> 3

* actually working v13 migration, v13 migration itest

* Address review

* sealing: Correct DDO snap pledge math

* itests: Mixed ddo itest

* pipeline: Fix sectorWeight

* sealing: convert market deals into PAMs in mixed sectors

* sealing: make market to ddo conversion work

* fix lint

* update gst

* Update actors and GST to lastest integ branch

* commit batcher: Update ProveCommitSectors3Params builder logic

* make gen

* use builtin-actors master

* ddo: address review

* itests: Add commd assertions to ddo tests

* make gen

* gst with fixed types

* config knobs for RequireActivationSuccess

* storage: Drop obsolete flaky tasts

---------

Co-authored-by: Jennifer Wang <jiayingw703@gmail.com>
Co-authored-by: Aayush <arajasek94@gmail.com>
Co-authored-by: Shrenuj Bansal <shrenuj.bansal@protocol.ai>
Co-authored-by: Phi <orjan.roren@gmail.com>
Co-authored-by: Andrew Jackson (Ajax) <snadrus@gmail.com>
Co-authored-by: TippyFlits <james.bluett@protocol.ai>

* feat: implement FIP-0063

* chore: deps: update to go-multiaddr v0.12.2 (#11602)

* feat: fvm: update the FVM/FFI to v4.1 (#11608) (#11612)

This:

1. Adds nv22 support.
2. Updates the message tracing format.

Co-authored-by: Steven Allen <steven@stebalien.com>

* AggregateProofType nil when doing batch updates

Use latest nv22 go-state-types version with matching update

* Update to v13.0.0-rc.2 bundle

* chore: Upgrade heights and codename

Update upgrade heights

Co-Authored-By: Steven Allen <steven@stebalien.com>

* Update epoch after nv22 DRAND switch

Update epoch after nv22 DRAND switch

* Update Mango codename to Phoneix

Make the codename for the Drand-change inline with Dragon style.

* Add UpgradePhoenixHeight to API params

* set UpgradePhoenixHeight to be one hour after Dragon

* Make gen

Make gen and UpgradePhoenixHeight in butterfly and local devnet to be in line with Calibration and Mainnet

* Update epoch heights (#11637)

Update epoch heights

* new: add forest bootstrap nodes (#11636)

Signed-off-by: samuelarogbonlo <sbayo971@gmail.com>

* Merge pull request #11491 from filecoin-project/fix/remove-decommissioned-pl-bootstrap-nodes

Remove PL operated bootstrap nodes from mainnet.pi

* feat: api: new verified registry methods to get all allocations and claims (#11631)

* new verireg methods

* update changelog and add itest

* update itest and cli

* update new method's support till v9

* remove gateway APIs

* fix cli internal var names

* chore:: backport #11609 to the feat/nv22 branch (#11644)

* feat: api: improve the correctness of Eth's trace_block (#11609)

* Improve the correctness of Eth's trace_block

- Improve encoding/decoding of parameters and return values:
  - Encode "native" parameters and return values with Solidity ABI.
  - Correctly decode parameters to "create" calls.
  - Use the correct (ish) output for "create" calls.
  - Handle all forms of "create".
- Make robust with respect to reverts:
  - Use the actor ID/address from the trace instead of looking it up in
    the state-tree (may not exist in the state-tree due to a revert).
  - Gracefully handle failed actor/contract creation.
- Improve performance:
  - We avoid looking anything up in the state-tree when translating the
    trace, which should significantly improve performance.
- Improve code readability:
  - Remove all "backtracking" logic.
  - Use an "environment" struct to store temporary state instead of
    attaching it to the trace.
- Fix random bugs:
  - Fix an allocation bug in the "address" logic (need to set the
    capacity before modifying the slice).
  - Improved error checking/handling.
- Use correct types for `trace_block` action/results (create, call, etc.).
  - And use the correct types for Result/Action structs instead of reusing the same "Call" action every time.
- Improve error messages.

* Make gen

Make gen

---------

Co-authored-by: Steven Allen <steven@stebalien.com>

* fix: add UpgradePhoenixHeight to StateGetNetworkParams (#11648)

* chore: deps: update to go-state-types v13.0.0-rc.1

* do NOT update the cache when running the real migration

* Merge pull request #11632 from hanabi1224/hm/drand-test

feat: drand quicknet: allow scheduling drand quicknet upgrade before nv22 on 2k devnet

* chore: deps: update to go-state-types v13.0.0-rc.2

chore: deps: update to go-state-types v13.0.0-rc.2

* feat: set migration config UpgradeEpoch for v13 actors upgrade

* Built-in actor events first draft

* itest for DDO non-market verified data w/ builtin actor events

* Tests for builtin actor events API

* Clean up DDO+Events tests, add lots of explainer comments

* Minor tweaks to events types

* Avoid duplicate messages when looking for receipts

* Rename internal events modules for clarity

* Adjust actor event API after review

* s/ActorEvents/Events/g in global config

* Manage event sending rate for SubscribeActorEvents

* Terminate SubscribeActorEvents chan when at max height

* Document future API changes

* More clarity in actor event API docs

* More post-review changes, lots of tests for SubscribeActorEvents

Use BlockDelay as the window for receiving events on the SubscribeActorEvents
channel. We expect the user to have received the initial batch of historical
events (if any) in one block's time. For real-time events we expect them to
not fall behind by roughly one block's time.

* Remove duplicate code from actor event type marshalling tests

Reduce verbosity and remove duplicate test logic from actor event types
JSON marshalling tests.

* Rename actor events test to follow go convention

Add missing `s` to `actor_events` test file to follow golang convention
used across the repo.

* Run actor events table tests in deterministic order

Refactor `map` usage for actor event table tests to ensure deterministic
test execution order, making debugging potential issues easier. If
non-determinism is a target, leverage Go's built-in parallel testing
capabilities.

* Reduce scope for filter removal failure when getting actor events

Use a fresh context to remove the temporary filter installed solely to
get the actor events. This should reduce chances of failure in a case
where the original context may be expired/cancelled.

Refactor removal into a `defer` statement for a more readable, concise
return statement.

* Use fixed RNG seed for actor event tests

Improve determinism in actor event tests by using a fixed RNG seed. This
makes up a more reproducible test suit.

* Use provided libraries to assert eventual conditions

Use the functionalities already provided by `testify` to assert eventual
conditions, and remove the use of `time.Sleep`.

Remove duplicate code in utility functions that are already defined.

Refactor assertion helper functions to use consistent terminology:
"require" implies fatal error, whereas "assert" implies error where the
test may proceed executing.

* Update changelog for actor events APIs

* Fix concerns and docs identified by review

* Update actor bundle to v13.0.0-rc3

Update actor bundle to v13.0.0-rc3

* Prep Lotus v1.26.0-rc1

- For sanity reverting the mainnet upgrade epoch to 99999999, and then only set it when cutting the final release

-Update Calibnet CIDs to v13.0.0-rc3

- Add GetActorEvents, SubscribeActorEvents, GetAllClaims and GetAllAllocations methods to the changelog

Co-Authored-By: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>

* Update CHANGELOG.md

Co-authored-by: Masih H. Derkani <m@derkani.org>

* Make gen

Make gen

* fix: beacon: validate drand change at nv16 correctly

* bump to v1.26.0-rc2

* test: cleanup ddo verified itest, extract steps to functions

also add allocation-removed event case

* test: extract verified DDO test to separate file, add more checks

* test: add additional actor events checks

* Add verification for "deal-activated" actor event

* docs(drand): document the meaning of "IsChained" (#11692)

* Resolve conflicts

I encountered multiple issues when trying to run make gen. And these changes fixed a couple of them:
- go mod tidy
- Remove RaftState/RaftLeader
- Revert `if ts.Height() > claim.TermMax+claim.TermStart || !cctx.IsSet("expired")` to the what is in the release/v1.26.0: `if tsHeight > val.TermMax || !expired`

* fixup imports, make jen

* Update version

Update version in master to v1.27.0-dev

* Update node/impl/full/dummy.go

Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>

* Adjust ListClaimsCmd

Adjust ListClaimsCmd according to review

---------

Signed-off-by: samuelarogbonlo <sbayo971@gmail.com>
Co-authored-by: TippyFlits <james.bluett@protocol.ai>
Co-authored-by: Aayush <arajasek94@gmail.com>
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
Co-authored-by: Jennifer Wang <jiayingw703@gmail.com>
Co-authored-by: Shrenuj Bansal <shrenuj.bansal@protocol.ai>
Co-authored-by: Andrew Jackson (Ajax) <snadrus@gmail.com>
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Samuel Arogbonlo <47984109+samuelarogbonlo@users.noreply.github.com>
Co-authored-by: LexLuthr <88259624+LexLuthr@users.noreply.github.com>
Co-authored-by: tom123222 <160735201+tom123222@users.noreply.github.com>
Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>
Co-authored-by: Masih H. Derkani <m@derkani.org>
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
2024-03-12 10:33:58 +01:00
e425322c2b misc: Drop the raft-cluster experiment 2023-11-29 17:28:46 +01:00
fbac220208 feat: chain node: Move consensus slasher to internal service 2023-08-01 12:06:18 -04:00
fe7cf0c39a feat: chain node: Move consensus slasher to internal service 2023-08-01 17:34:44 +02:00
db8b593c9b make MsgIndex configurable, disabled by default. 2023-03-13 12:01:04 +02:00
171734ea31 hook the index into the rest of lotus 2023-03-12 15:25:07 +02:00
f291572b4a Merge branch 'release/v1.20.0' into asr/merge-release-into-master 2023-02-03 09:27:04 -05:00
b46258d0bf feat: evm: align events implementation with FIP-0049 and FIP-0054. (#10152)
- Event keys are now t1, t2, t3, t4 for topics; and d for data.
- ref-fvm no longer stores events in the blockstore for us. It just
  returns events to the client, who is now responsible for handling
  them as it wishes / according to its configuration.
- Add a flag to VMOpts to have the events AMT be written in the blockstore.
- Add a flag to the ChainStore to advertise to the rest of the system
  if the ChainStore is storing events.
- Enable that flag if the EthRPC is enabled (can also add an explicit
  configuration flag if wanted).
2023-02-02 19:10:30 -08:00
8ce0b984b7 itests: Fix TestEthFilterAPIDisabledViaConfig 2023-02-02 15:59:55 +01:00
3d207de6aa node builder: Use gateway eth module in lite mode 2023-02-02 13:39:40 +01:00
4f199ada40 Merge branch 'release/v1.20.0' into asr/merge-release-into-master 2023-01-20 17:32:29 -05:00
6601d9031d Set default path for eth event db, set it to enabled by default if EnableEthRPC is set 2023-01-19 18:40:28 -05:00
eaccb571a5 fix: config: Fix eth rpc typo (#10076) 2023-01-19 20:35:19 +00:00
66f5ee4ae9 config: Fevm.EnableEthPRC 2023-01-19 18:31:17 +01:00
e715384173 Merge branch 'release/v1.20.0' into asr/merge-release-into-master 2023-01-16 10:51:22 -05:00
f8dee0983a review fixes 2023-01-16 01:56:55 -05:00
a8436074a6 Store mapping from hashes for Ethereum transactions to Filecoin Message Cids 2023-01-16 01:17:56 -05:00