1317 Commits

Author SHA1 Message Date
945451c528 feat(miner): remove batch balancer-related functionality (#12919)
Closes: https://github.com/filecoin-project/lotus/issues/12902
2025-03-19 16:07:32 +11:00
632e20a626 feat!: fip-0100 dev builds of builtin-actors & go-state-types
Changes some APIs exposed in the miner actor wrapper.
2025-03-19 13:43:52 +11:00
5307183f45 chore: update go-state-types with empty big.Int{} handling (#12936) 2025-03-10 20:49:41 +00:00
4c9687acc2 fix(docs): minor godoc corrections (#12843)
Signed-off-by: petercover <raowanxiang@outlook.com>
2025-01-23 01:05:25 +00:00
f422705801 chore(docs): fix minor typos (#12820) 2025-01-10 05:06:27 +00:00
bbd53a6af7 chore: use time.Since instead of time.Now().Sub (#12801) 2024-12-30 14:13:53 +11:00
a704e4710d chore: delete STM annotations (#12752)
Signed-off-by: Jakub Sztandera <oss@kubuxu.com>
2024-12-05 16:45:07 +11:00
3f31b3b423 chore: fix struct name in comment (#12689)
Signed-off-by: pinglanlu <pinglanlu@outlook.com>
2024-11-15 17:50:57 +11:00
61ca37b861 chore: update triplewz/poseidon (#12602) 2024-10-16 15:13:17 +00:00
06bec3a72f feat(miner): add DDO-friendly StateMinerInitialPledgeForSector (#12384)
* feat(miner): add DDO-friendly StateMinerInitialPledgeForSector

Fixes: https://github.com/filecoin-project/lotus/issues/12369

deprecate StateMinerInitialPledgeCollateral since it only accounts for deals
in PCI, which aren't present in a DDO world

* feat: simplify StateMinerInitialPledgeForSector arguments

* feat(miner): use StateMinerInitialPledgeForSector for collateral calcs

* docs: add StateMinerInitialPledgeForSector to CHANGELOG

* fix: undo unnecessary docsgen changes

* chore(state): use types.EmptyInt as err return instead of big.Zero()

* chore(miner): add tests for pledge calculations

* doc(miner): more docs about pledge calculation delta

* chore(miner): tweaks to pledge api from feedback
2024-09-25 16:51:55 +10:00
312fa3a5cc Simplify code generation logic for a faster and better UX (#12493)
The changes here simplify the string of targets executed as part of
`make gen`. The simplifications include:
* parallelised code generation whenever possible
* re-implementation of documentation generation for a significant
  simplification and improved readability.
* unified mocks generation to avoid multiple slow calls to `go run` per
  package.

Note, the changes introduced here are purely mechanical and do not alter
the Lotus node functionality.

Fixes #8392
2024-09-20 16:32:54 +00:00
a83d02efba chore: deps: bump xerrors / fix lint (#12438) 2024-09-07 21:43:15 +00:00
e31e4e8522 chore: tests: fix failures on recent macos
On modern darwin os.ReadDir("/dev/fd") always results in an error. Because of
this storage/sealer/ffiwrapper/sealer_test.go invariably fails.

Instead of fixing the FD enumerator, remove the check entirely, as it has been
disabled since https://github.com/filecoin-project/lotus/pull/7420/files
2024-09-02 20:24:34 +10:00
a5cb674bb1 fix: error check 2024-08-20 10:06:48 +02:00
415e7e600c fix: miner: Fix DDO pledge math (#12341)
* Power is units of Space * Time so multiply by deal duration

* fix: miner: Fix DDO pledge math

* appease the changelog checker

* Fix gen

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
2024-08-13 14:34:08 +02:00
0b94fee709 chore: convert everything relevant to go-commp-utils/v2 2024-08-13 11:33:22 +10:00
a5f93216dc chore(docs): fix some function names (#12368)
Signed-off-by: luozexuan <fetchcode@139.com>
2024-08-12 02:50:16 +00:00
8dd9a555aa chore: sealing: instead of variable use method to get miner sector expiration 2024-08-12 12:39:25 +10:00
10f6976a55 chore: docs: fix some misspellings (#12333)
Signed-off-by: riskrose <riskrock@qq.com>
2024-08-02 03:18:16 +00:00
2ad743aca2 fix: lotus-miner: remove provecommit1 method (#12251)
* remove provecommit1

* add changelog

* update precommit and commit params

* fix lint error

* fix commit params
2024-07-24 19:02:29 +02:00
1bc8a8b32c chore: remove (deprecated) deps on build/ proxy-constants
This is a large diff, yet should have exactly zero functional changes

Ideally as a result of this some parts of the depchain will become lighter,
with downstream reaping the same benefits as the team that initiated this split.

P.S. work was done while forming better intuition of current dependency graph
2024-07-22 17:36:12 +04:00
526d1ffe82 chore: lint: enable type godoc linting 2024-07-19 11:10:11 +10:00
428524fdd8 chore: lint: enable method godoc linting 2024-07-19 11:10:11 +10:00
7635d55412 NewCommitBatcher now has an additional
error return to deal with errors arising from fetching the sealing config.
2024-07-16 13:06:05 +04:00
85587dd5b4 fix!: sealer: handle initialisation error without panic
storage/pipeline.NewPreCommitBatcher and storage/pipeline.New now have an additional
error return to deal with errors arising from fetching the sealing config.
2024-07-16 10:24:45 +04:00
711bb7db3f chore!: markets: remove stray unixfs constants, features and references 2024-07-12 08:57:01 +04:00
286fadaca4 nv23 deps (#12129)
* deps: ffi: upgrade filecoin-ffi to v1.28.0-rc1

* deps: types: update go-state-types to v0.14.0-rc1

* fix: sealer: define SDR layers for NI-PoRep
2024-06-21 10:41:20 +04:00
e3deda0b2b cleanup: miner: remove markets and deal-making from Lotus Miner (#12005)
* remove client CLI

* remove markets CLI from miner

* remove markets from all CLI

* remove client API

* update go mod

* remove EnableMarkets flag

* remove market subsystem

* remove dagstore

* remove index provider

* remove graphsync and data-transfer

* remove markets

* go mod tidy

* fix cbor gen deps

* remove deal making from config

* remove eol alert

* go mod tidy

* changes as per review

* make jen

* changes as per review

* merge master

* remove libp2p from config

* miner does not have libp2p conn in api test
2024-06-05 18:14:50 +04:00
41ebef80e6 fix: storage: Fix a race in GenerateWindowPoStAdv (#12064) 2024-06-03 18:46:59 +00:00
1879570628 chore: fix function names (#12043)
Co-authored-by: tomfees <170385392+tomfees@users.noreply.github.com>
2024-05-27 03:51:55 +00:00
c1f99c58c5 feat: curio ffiselect: Isolate gpu calls in a subprocess (#11994)
* started

* so far so good

* builds and looks good

* changing level of abstration. some work remains

* it builds

* betterment

* import order

* 2

* stupid linter - you can cast a nil

* build commit and date

* nicer

* tmp and nide makefile

* comments handled

* oops

* added debug and reg

* ffiselect: change err encode to strings, fix some bugs

* ffiselect: Wrap rust logs into go-log

* ffiselect: Make the linter happy

* verification tests

* ffiselect: Fix startup

---------

Co-authored-by: Łukasz Magiera <magik6k@gmail.com>
2024-05-23 13:03:59 +02:00
7e3846c669 feat: curio: allow multiple pieces per sector (#11935)
* multiple piece per sector, DDO deals

* in memory to DB

* sql parser

* add seal command

* multi piece TreeD

* redo filler pieces

* remove psql exception handling

* fix deal sectors porep

* fix tests

* ddo deals

* lower SDR cpu for test

* devnet cpu 0

* get params for itest

* fix itest sector size

* revert sdr devnet cpu

* improve SectorStatus API

* account for verified constraints
2024-05-22 10:08:32 +02:00
3ee09115e6 ci: deprecate circle ci in favour of github actions (#11786) 2024-05-16 13:31:08 +02:00
26d3fd2ecc chore: lint: fix lint errors with new linting config
Ref: https://github.com/filecoin-project/lotus/issues/11967
2024-05-13 14:52:09 +10:00
441f5995d3 fix: curio: node UI & darwin gpu count (#11950)
* fix: curio: Update pgx imports, fix db_storage alloc

* feat: curioweb: Improve task_history indexes (#11911)

* 1

* relatable

* add and delete layer

* chore: curio: merge master (#11956)

* Fixing dead links (#11907)

* ci: ci: create gh workflow that updates sorted pr checks (#11861)

* ci: create gh workflow that updates sorted pr checks

* ci: use grouped_by_result template for pr checks sticky comment

* chore: apply pr review suggestion

* Avoid cfg lookup on chain remove since unenabled splitstore delete is noop anyway (#11916)

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* Fix mismatched method names in comments (#11913)

Signed-off-by: forcedebug <forcedebug@outlook.com>

* release: v1.26.3 (#11908) (#11915) (#11922)

* deps: update dependencies to address migration memory bloat

to address memory concerns during a heavy migration

Ref: https://github.com/filecoin-project/go-state-types/pull/260
Ref: https://github.com/whyrusleeping/cbor-gen/pull/96
Ref: https://github.com/filecoin-project/go-amt-ipld/pull/90

* release: prep v1.26.3 patch

Prep v1.26.3 patch release:
- Update changelog, version and make gen + make docsgen-cli

* deps: update cbor-gen to tagged version

deps: update cbor-gen to tagged version

* deps: update go-state-types to tagged version

deps: update go-state-types to tagged version v0.13.2

* chore: deps: update go-state-types to v0.13.3

Fixes a panic when we have fewer than 1k proposals.

---------

Co-authored-by: Phi-rjan <orjan.roren@gmail.com>
Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Steven Allen <steven@stebalien.com>

* Refactor `LookupID*` APIs in `StateManager` and `StateTree`

The naming of `LookupID` can cause confusion when resolving actor IDs vs
 ID addresses. To avoid this:

* Refactor `StateTree` `LookupID` to `LookupIDAddress`, because it
returns ID address.
* Refactor `StateManager` `LookupID` to
`LookupIDAddress` because it also returns ID address via a chain call to
`StateTree`.
* Introduce a new API `StateManager` dedicated to resolving address to
actor ID, called `LookupID` which returns `abi.ActorID`.

For context, see:
 * https://github.com/filecoin-project/lotus/pull/11723#discussion_r1534728607

* Add v13 support to invariants-checker (#11931)

Add v13 support to invariants-checker

* chore: docs: nv-skeleton documentation (#11065)

* nv-skeleton documentation

Add a tutorial for how one can create a nv-skeleton in Lotus

* Add footnote for `Add migration` step

Add footnote for `Add migration` step

* Indent migration-code

Indent migration-code to make it show properly as a footnote.

* Add ref-fvm and filecoin-ffi checklist

Add ref-fvm and filecoin-ffi checklist

* Add Filecoin-FFI steps

Add Filecoin-FFI steps

* Add step to params_butterfly.go

Add step to params_butterfly.go

* Fix typo

Fix typo

* Add links to reference PRs

Add links to reference PRs

* Update ref-fvm list

Update ref-fvm list

* feat: curio: add StorageInit api (#11918)

* feat: add StorageInit api

* remove unused variables

* fix gen check

* feat: curio: simpler reservation release logic (#11900)

* simpler release logic

* oops, plus simpler

* simpler

* fix NewLine (#11893)

* fix(events): check for sync-in-progress (#11932)

* feat(events): adjust indexes in event index db to match query patterns

Introduces a v4 migration that just adjusts indexes.

Copies some improvements from https://github.com/filecoin-project/lotus/pull/11723

Closes: https://github.com/filecoin-project/lotus/issues/11909

* fix(pipeline): should return if error occurred when get network version (#11902)

* fix(events): correct log msg for v4 events index db migration

* chore: remove duplicate words in strings and comments

* fix(events): register events index db migration v4

* fix: curio seal: Failed commit retry strategy (#11870)

* ffi: improved-error-handling

* curio seal: Failed commit retry strategy

* use master ffi

* mod tidy

* fix: curio: Update pgx imports, fix db_storage alloc

* feat: curioweb: Improve task_history indexes (#11911)

* mod tidy

* Event index should be unique for tipsets (#11952)

* event index should be unique for tipsets

* fix formatting

* migrate to version 5

* chore: bump build version in master (#11946)

* Bump version

Bump version in master branch in preperation for cutting v1.27.0-rc1

* chore: bump build-version

chore: bump build-version

* feat: curioweb: Show piece info on the sector page (#11955)

* curio: feat: break trees task into TreeD(prefetch) and TreeRC (#11895)

* break trees task

* fix TreeD reservation

* fix nil pointer err

* apply suggestions

* fix allocate file types

* fix dbIndex inserts

* set resource, move release func

* refactor func(), update memory

* remove extra release

---------

Signed-off-by: forcedebug <forcedebug@outlook.com>
Co-authored-by: parthshah1 <parthhshah171@gmail.com>
Co-authored-by: Piotr Galar <piotr.galar@gmail.com>
Co-authored-by: ZenGround0 <5515260+ZenGround0@users.noreply.github.com>
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
Co-authored-by: forcedebug <167591285+forcedebug@users.noreply.github.com>
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Phi-rjan <orjan.roren@gmail.com>
Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: Masih H. Derkani <m@derkani.org>
Co-authored-by: Lee <octalgah@gmail.com>
Co-authored-by: Andrew Jackson (Ajax) <snadrus@gmail.com>
Co-authored-by: beck <34204218+beck-8@users.noreply.github.com>
Co-authored-by: 0x5459 <0x5459@protonmail.com>
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
Co-authored-by: Łukasz Magiera <magik6k@gmail.com>
Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>

* linter oops

* gen cleanup

* fix

* named returns are confusing

---------

Signed-off-by: forcedebug <forcedebug@outlook.com>
Co-authored-by: Łukasz Magiera <magik6k@gmail.com>
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
Co-authored-by: LexLuthr <88259624+LexLuthr@users.noreply.github.com>
Co-authored-by: parthshah1 <parthhshah171@gmail.com>
Co-authored-by: Piotr Galar <piotr.galar@gmail.com>
Co-authored-by: ZenGround0 <5515260+ZenGround0@users.noreply.github.com>
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
Co-authored-by: forcedebug <167591285+forcedebug@users.noreply.github.com>
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Phi-rjan <orjan.roren@gmail.com>
Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: Masih H. Derkani <m@derkani.org>
Co-authored-by: Lee <octalgah@gmail.com>
Co-authored-by: beck <34204218+beck-8@users.noreply.github.com>
Co-authored-by: 0x5459 <0x5459@protonmail.com>
Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>
2024-05-04 10:35:30 +02:00
00edad4e4d curio: feat: break trees task into TreeD(prefetch) and TreeRC (#11895)
* break trees task

* fix TreeD reservation

* fix nil pointer err

* apply suggestions

* fix allocate file types

* fix dbIndex inserts

* set resource, move release func

* refactor func(), update memory

* remove extra release
2024-05-02 11:17:33 +02:00
1b6bffd339 fix: curio: Update pgx imports, fix db_storage alloc 2024-04-30 20:43:56 +02:00
15f5f4745d fix(pipeline): should return if error occurred when get network version (#11902) 2024-04-29 12:21:40 +02:00
5e1d8f661e feat: curio: simpler reservation release logic (#11900)
* simpler release logic

* oops, plus simpler

* simpler
2024-04-26 11:11:42 +02:00
103a7e0697 apply suggestions 2024-04-19 16:07:01 +02:00
72fac27a4d account for cc sectors 2024-04-19 16:07:01 +02:00
c8dec8de65 add allocation check for commit 2024-04-19 16:07:01 +02:00
6b3e9b109f feat: curio: sectors UI (#11869)
* cfg edit 1

* jsonschema deps

* feat: lp mig - first few steps

* lp mig: default tasks

* code comments

* docs

* lp-mig-progress

* shared

* comments and todos

* fix: curio: rename lotus-provider to curio (#11645)

* rename provider to curio

* install gotext

* fix lint errors, mod tidy

* fix typo

* fix API_INFO and add gotext to circleCI

* add back gotext

* add gotext after remerge

* lp: channels doc

* finish easy-migration TODOs

* out generate

* merging and more renames

* avoid make-all

* minor doc stuff

* cu: make gen

* make gen fix

* make gen

* tryfix

* go mod tidy

* minor ez migration fixes

* ez setup - ui cleanups

* better error message

* guided setup colors

* better path to saveconfigtolayer

* loadconfigwithupgrades fix

* readMiner oops

* guided - homedir

* err if miner is running

* prompt error should exit

* process already running, miner_id sectors in migration

* dont prompt for language a second time

* check miner stopped

* unlock repo

* render and sql oops

* curio easyMig - some fixes

* easyMigration runs successfully

* lint

* part 2 of last

* message

* merge addtl

* fixing guided setup for myself

* warn-on-no-post

* EditorLoads

* cleanups and styles

* create info

* fix tests

* make gen

* sector early bird

* sectors v2

* sector termination v1

* terminate2

* mjs

* minor things

* flag bad sectors

* fix errors

* add dealweight and deals

* change column width

* refactor sql, handle sealing sectors

* fix estimates

---------

Co-authored-by: LexLuthr <88259624+LexLuthr@users.noreply.github.com>
Co-authored-by: LexLuthr <lexluthr@protocol.ai>
Co-authored-by: LexLuthr <lexluthr@curiostorage.org>
2024-04-18 14:57:29 -05:00
52a190aa7b chore: fix some typos in comments (#11892)
Signed-off-by: writegr <wellweek@outlook.com>
2024-04-18 10:34:20 +02:00
c785e59371 feat: curio: storage index gc task (#11884)
* curio storage path gc: lay out the structure

* curio gc: Implement storage metadata gc

* move bored singleton task impl to harmonytask

* curio: run storage gc task on storage node

* make gen
2024-04-16 16:34:48 -05:00
3ebb92a976 fix: curio: fix incorrect null check for varchar column (#11881)
* err check empty string

* log skipped tasks
2024-04-15 19:27:31 -05:00
bc43bd6d69 fix: local storage reservations fixes (#11866)
* paths: Debugging local storage reservations

* paths: Log when individual reservation is less than on-disk space

* paths: fix debug reservations print

* paths: More reserve logs

* paths: More more reserve logs

* paths: add stacks to duplicate done call log

* curio: task storage: Release storage at most once

* curio: cleanup before restarting sdr

* address review

* paths: Simplify reservation release logic
2024-04-15 17:33:06 -05:00
3cc62e04e9 feat: curio: Cleanup data copies after seal ops (#11847)
* feat: curio: Cleanup data copies after seal ops

* curio: make ensureOneCopy safer

* curio: Don't randomly remove cache files

* Address review
2024-04-11 11:35:09 -05:00
d7d849cf20 feat: curioweb: Sector info page (#11846)
* feat: curioweb: Sector info page

* Address review
2024-04-11 11:30:53 -05:00
dd6fef807b fix: sealing: Use PRU3 After Nv22 (#11831) 2024-04-05 20:58:00 +02:00