Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upstream: Prague code merge [v1.13.15, v1.14.11] #2761

Merged
merged 584 commits into from
Nov 26, 2024

Conversation

buddh0
Copy link
Collaborator

@buddh0 buddh0 commented Nov 22, 2024

Description

Prague code merge [v1.13.15, v1.14.11(latest)]

Rationale

One of the preparations for activating the Prague hard fork on bsc, status
✅ compile pass

make all

✅ fix all CIs
✅ mine blocks with node-deploy script

todo:

  1. code review and fix
  2. various nodes and sync test
  3. QA regression testing

Example

add an example CLI or API response...

Changes

Major Changes

eth: make transaction propagation paths in the network deterministic (#29034)
core/state: parallelise parts of state commit (#29681)
Load trie nodes concurrently with trie updates, speeding up block import by 5-7% (#29519#29768#29919).
core/vm: reject contract creation if the storage is non-empty (#28912)
Add state reader abstraction (#29761)
Stateless witness prefetcher changes (#29519)
not follow changes with trie_prefetcher, the implemenation in bsc is very different
core: use finalized block as the chain freeze indicator (#28683)
in bsc, this feature only enabled with multi-database

New EIPs

core/vm: enable bls-precompiles for Prague (#29552)
EIP-2935: Serve historical block hashes from state (#29465)

Clear Up

eth, eth/downloader: remove references to LightChain, LightSync (#29711)
eth/filters: remove support for pending logs(#29574)
Drop large-contract (500MB+) deletion DoS protection from pathdb post Cancun (#28940).
Remove totalDifficulty field from RPC, in accordance with spec update#30386

Merged but Reverted

consensus, cmd, core, eth: remove support for non-merge mode of operation (#29169)
miner: refactor the miner, make the pending block on demand (#28623)
miner: lower default min miner tip from 1 gwei to 0.001 gwei(#29895)
bsc only has tip, 1 Gwei is the min value now
eth/downloader: purge pre-merge sync code (#29281)
all: remove forkchoicer and reorgNeeded (#29179)

Flags and Configure changes

deprecated flags
  • light.serve
  • light.ingress
  • light.egress
  • light.maxpeers
  • light.nopruning
  • light.nosyncserve
  • metrics.expensive
delete fields in config.toml

[Eth]
LightPeers
[Eth.GPO]
Default

Others

all: update to go version 1.23.0 (#30323)
Switch to using Go's native log/slog package instead of golang/exp (#29302).
Add the geth db inspect-history command to inspect pathdb state history (#29267).
Improve the discovery protocol's node revalidation (#29572#29864#29836).
Blobpool related flags in Geth now actually work. (#30203)
core/rawdb: implement in-memory freezer (#29135)

more details to refer release note between [v1.13.15, v1.14.11] in geth

fjl and others added 30 commits June 5, 2024 19:31
enode.Node was recently changed to store a cache of endpoint information. The IP address in the cache is a netip.Addr. I chose that type over net.IP because it is just better. netip.Addr is meant to be used as a value type. Copying it does not allocate, it can be compared with ==, and can be used as a map key.

This PR changes most uses of Node.IP() into Node.IPAddr(), which returns the cached value directly without allocating.
While there are still some public APIs left where net.IP is used, I have converted all code used internally by p2p/discover to the new types. So this does change some public Go API, but hopefully not APIs any external code actually uses.

There weren't supposed to be any semantic differences resulting from this refactoring, however it does introduce one: In package p2p/netutil we treated the 0.0.0.0/8 network (addresses 0.x.y.z) as LAN, but netip.Addr.IsPrivate() doesn't. The treatment of this particular IP address range is controversial, with some software supporting it and others not. IANA lists it as special-purpose and invalid as a destination for a long time, so I don't know why I put it into the LAN list. It has now been marked as special in p2p/netutil as well.
rlp: no need to repeat calling len
Fixes an issue where discovery responses were not recognized.
* cmd/utils, consensus/beacon, core/state: when configured via stub  flag: prefetch all reads from account/storage tries, terminate prefetcher synchronously.

* cmd, core/state: fix nil panic, fix error handling, prefetch nosnap too

* core/state: expand prefetcher metrics for reads and writes separately

* cmd/utils, eth: fix noop collect witness flag

---------

Co-authored-by: Péter Szilágyi <[email protected]>
* .golangci.yml: enable check for consistent receiver name

* beacon/light/sync: fix receiver name

* core/txpool/blobpool: fix receiver name

* core/types: fix receiver name

* internal/ethapi: use consistent receiver name 'api' for handler object

* signer/core/apitypes: fix receiver name

* signer/core: use consistent receiver name 'api' for handler object

* log: fix receiver name
* fix: Optimize regular initialization

* modify var name

* variable change to private types
…rom trie Commit (#29869)

* core/state, eth/protocols, trie, triedb/pathdb:  remove unused error return from trie Commit

* move set back to account-trie-update block scoping for easier readability

* address review

* undo tests submodule change

* trie:  panic if BatchSerialize returns an error in Verkle trie Commit

* trie: verkle comment nitpicks

---------

Co-authored-by: Péter Szilágyi <[email protected]>
* beacon/light/request: add server test for event after unsubscribe

* beacon/light/api: fixed double stream.Close()

* beacon/light/request: add checks for nil event callback function

* beacon/light/request: unlock server mutex while unsubscribing from parent
* upgrade lock usage

* revert unnecessary change
updates some docstrings
---------

Co-authored-by: rjl493456442 <[email protected]>
holiman and others added 9 commits September 30, 2024 15:39
This PR removes the dependencies on `lightchaindata` db as the light
protocol has been deprecated and removed from the codebase.
* fix compile for statedb_test.go

* format addresses of core/vm/contracts

* fix compilation of package simulated

* tests: update tests

* remove rootParent in triePrefetcher

* add ClearAccessList for processing system txs before Cancun

* copy prefetcher when copy statedb

* always prefetch the account trie once the prefetcher is constructed

* recover TriePrefetchInAdvance

* fix comment for s.StopPrefetcher

* fix noTrie in statedb.go

* improve SnapToDiffLayer

* fix test cases in core/state

* fix compilation of test cases in core

* fix compilation of test cases in cmd, internal and trie

* remove Requests in BlockBody

* define RequestsHash in parlia engine

* fix compilation of test cases in eth

* fix compilation of test cases in ethclient and miner

* fix compilation of test cases in triedb/pathdb

* add todo for relation between witness and noTrie

* fix Nancy CI

* fix golangci-lint part I

* update golang to 1.23.x from 1.21.x in workflows

* remove sepoliaAllocData and holeskyAllocData

* modifiy interface FinalizeAndAssemble

* fix golangci-lint part II

* fix golangci-lint part III

* fix golangci-lint done

* check for multidb

* improve interface EnableBlockValidator

* fix getGenesisState

* code review for core/rawdb

* fix mustCopyTrie

* fix state_processor

* adapt for changes in func NewBlock

* improve EncodeSigHeader

* fix test cases in dir core, part I

* fix TestParliaBlobFeeReward

* fix repairHistory

* remove diskdb in triedb.Database

* fix asyncnodebuffer

* adapt memory freezer for blob table

* comment verkle related test cases

* fix golang lint

* use 1.21.x to run golangci-lint

* fix TestBsc1Messages

* fix TestSnapSyncWithBlobs

* fix traceTx

* update golang version for docker

* improve newWorker

* fix: truncate state history ut

* TestTailTruncateHistory clear up

* fix test cases in dir tests

* remove deprecated flags in truffle test

* comment filterNode

* HeaderChain.GenesisHeader

* fix golangcilint

* fix truffle-test CI

* update .nancy-ignore to avoid conflicts with develop branch

* update diff go-ethereum patch

* remove isTTDReached in worker

* fix mockparlia FinalizeAndAssemble

* fix golangci-lint

* fix makeEnv

* change log level for tri prefetch

---------

Co-authored-by: joeylichang <[email protected]>
Co-authored-by: joey <[email protected]>
@buddh0 buddh0 marked this pull request as ready for review November 22, 2024 07:23
@buddh0 buddh0 changed the base branch from master to develop November 22, 2024 07:25
@buddh0
Copy link
Collaborator Author

buddh0 commented Nov 22, 2024

ignore Lint Commit CI, because lots of commits upstreamed

core/tracing/hooks.go Outdated Show resolved Hide resolved
func (tab *Table) delete(node *node) {
tab.mutex.Lock()
defer tab.mutex.Unlock()
// TODO(Matus): fix the filterNode feature
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

p2p module changes a lot,
I tried to put filterNode logic into new codes, but failed
@MatusKysel please help recover the filterNode feature

triedb/pathdb/difflayer.go Outdated Show resolved Hide resolved
@@ -3,7 +3,6 @@ NetworkId = 714
SyncMode = "full"
NoPruning = false
NoPrefetch = false
LightPeers = 100
Copy link
Collaborator Author

@buddh0 buddh0 Nov 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

* eth/tracers: add onSystemTxEnd hook for system tx

* core/tracing/hooks.go: add notes
@maoueh
Copy link

maoueh commented Nov 25, 2024

Re-linking my comment #2768 (comment) here for increased visibility.

consensus/parlia/parlia.go Show resolved Hide resolved
core/block_validator.go Show resolved Hide resolved
core/blockchain.go Show resolved Hide resolved
core/state/snapshot/snapshot.go Show resolved Hide resolved
core/state/snapshot/snapshot.go Show resolved Hide resolved
core/state_transition.go Show resolved Hide resolved
@zzzckck zzzckck merged commit c49ea92 into develop Nov 26, 2024
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.