Before we get into the planning of how ReGenesis can be implemented and rolled out, here is a quick summary of what it is. Although the idea is not new, it has been proposed in the current form here: https://ethresear.ch/t/regenesis-resetting-ethereum-to-reduce-the-burden-of-large-blockchain-and-state/7582
There is also an explainer post: https://medium.com/@mandrigin/regenesis-explained-97540f457807
ReGenesis logically splits the Ethereum state into two disjoint parts - active state and inactive state. There are ReGenesis “events” occuring regularly, at pre-scheduled times, but not very frequently, for example, every 1 million blocks, which is roughly 6 months. At a ReGenesis event, the active state is reset to just the state root hash, and the entire state is considered inactive. After that, until the next ReGenesis event, new state items get added to the active state, existing state items in the inactive state are moved from inactive into the active state (activated), and there, in the active state, they may change their values.
The only two ways to activate a piece of state, or add new data into the active state are transaction witness and block witness.
Transactions get a new attribute, witness, which is a string of bytes encoding a collection of merkle multiproofs. Any part of the these multiproofs that are still valid against the active state, are added to the active state before transaction is executed. (TODO: Explain more how and why only part of these multiproofs will be valid most of the time, and how to find the valid parts).
Blocks also get a new attribute, witness, which has the same encoding and capabilities as transaction witnesses, except that the valid parts of the block witness are added to the active state before executing any transaction in the block. One reason block witnesses are required is activation of miner’s coinbase address in the state. Another reason is ensuring a more gradual transition to the ReGenesis. For example, miner may choose to make an otherwise failing transaction succeed by identifying pieces of state missing from the transaction’s witnesses, and adding these pieces into the block witness (transaction repair).
Depending on where we are in the process of introducing ReGenesis, the extra bytes in the transaction witnesses and block witnesses are either free of gas charge (initial stages), and later charged from transaction sender (for transaction witness ony).
The semantics of executing transactions changes slightly. Now whenever transaction attempts to read a part of state, but that part of state is inactive, transaction reverts and gas spent up to that point is consumed.
Another change in the semantics is that if at the end of the block’s execution, the miner’s coinbase is not in the active state, the mining rewards are not given (reward and transaction fees are essentially burnt).
After each ReGenesis event, the chainID changes to a new value. This ensures that any transactions that were created before the ReGenesis event and are still “in flight”, will not simply fail on chain (because they will most probably have insufficient witness). Instead, transactions for the new ReGenesis “epoch” need to be explicitely recreated. This would help transaction creators have more robust processes for the ReGenesis events.
The hope is that by re-setting the active state regularly, and requiring that only the active state is accessible by transactions, we can keep the active state small enough so that Ethereum nodes can store it in RAM, or at least on very low-latency devices. This, in turn, will ensure that the actual cost of state accessing operations (
EXTCODEHASH, etc.) does not keep increasing. That means we can safely increase block gas limit to much higher values that we have today.
ReGenesis makes it more explicit at which point any node can sync “from scratch”. This means that it will become a norm to have an implementation that does not implement all the historical rules for all Ethereum forks. Instead, only behaviour since the last ReGenesis is a must, and all historical rules can be moved into specialised historical tracing tools.
Transactions will become larger because they will need to include witnesses. We expect this extra weight to be the highest right after a ReGenesis event, when transactions will need to activate the parts of the state they are using. However, active state can saturate to some degree, and more and more transactions will not need to include any witnesses, or very small witnesses.
Altough the burden on the core Ethereum infrastructure is likely to decrease, the burden on transaction-sending intfrastructure will increase. In order to compose a successful transaction, one would potentially need access not just to active state, but also to inactive state. In order to assist with this task, we would like network participants to create state snapshot in suitable formats and distribute them over BitTorrent (and perhaps other networks). In the same way, sets of historical blocks and receipts will be distributed for the purpose of retracing past transactions.
ReGenesis changes the status quo in that more aggressive optimisations will need to be done on the boundaries of the Ethereum network (where transactions are being created and injected), rather than in the core (critical infrastructure, “center” of the network). It may be more fair because boundaries are usually the places where the value is captured. Also, there is more diversity and decentralisation on the boundaries (there are more users than critical node operators). Failure to optimise on the boundaries does not represent systemic risk (i.e. it does not bring down the entire system). In the status quo, failure to optimise (in the core of the network) does represent systemic risk, and can potentially bring the entire system down.
Broadly, the proposed transition from where we are to ReGeneis is via a soft-fork, followed by a hard fork. After the soft-fork, most of the benefits for the core infrastructure can be made tangible, at the cost of running some amount of altruistic (or sponsored) nodes. Following hard-fork is required to bring about the extra costs and downsides, therefore removing the reliance on the altruistic or sponsored nodes.
Soft forks can be seen as extra restrictions on the validity of transactions and blocks. Some blocks and transactions that are valid before a soft fork, are treated as invalid under the soft fork. Soft forks have been used in Bitcoin (most famously to activate SegWit), but were deemed to be unworkable on Ethereum after an attempt to do one to restrict the transactions to the DAO in 2016: https://hackingdistributed.com/2016/07/05/eth-is-more-resilient-to-censorship/
The main reason soft forks are deemed to be unworkable on Ethereum is that transactions can be well formed, and funded, and thus propagating through the transaction pool, but then hit the miners, where they would need to be fully executed only to find out that they are invalid.
In order to overcome this challenge, the plan is to gradually create two auxillary networks:
This transaction pool can be created as a devp2p subprotocol, or as a completely new network. It will serve the same purpose as the the transaction pool today, but with some extra restrictions and extra capabilities. Any transaction message relayed on this network will need to include witness information. This witness needs to have a root at any state root between the latest ReGenesis event and the latest current block. This witness needs to be sufficient to verify sender’s balance, nonce, and also fully execute the transaction without having any access to the Ethereum state. If there is no witness, or it does not satisfy the requirements, the transaction is deemed invalid and gets removed from the pool. There could be some extra temporary penalty logic to prevent the same account regularly creating invalid transactions and waste tx pool network’s resources. At any point in time, the witness of a transaction can be updated by anyone, to include more recent information and rely on a more recent state root (but not the other direction, updates to more distant root are not permitted), as long the update does not remove previously included items. This update mechanism is used to knock out the transactions that have invalid nonce because the same sender had some other transactions included into the blocks, or insufficient funds because they have been spent by the intervining transactions.
There are few benefits of this network:
The transaction pool network can be bootstrapped by creating network bridge nodes. These bridge nodes connect to both
eth network and tx pool network, and relay transaction from one to another. If the idea proves successful, transaction injection will gradually move from
eth network, and wallets, exchanges, and dapp backends will start injecting transactions directly into the tx pool network.
It is the main mechanism for the enforcement of the soft fork. Here, Ethereum nodes will run a variant of Avalanche protocol to quickly agree on the composition of the blocks to be mined by the miners. Blocks agreed by the pre-consensus network will not include transactions that would, under ReGenesis rules, have to be accompanied by a very large witness, so that either the sending account would not have enough funds to pay for it, or the block gas limit would not allow it. This is similar to what the transaction pool network is doing, but while transaction pool provides only coarse-grained enforcement, the pre-consensus network provides a fine-grained enforcement, because transactions are evaluated in the context of being run in a proper block. Once a block is prepared and agreed by the pre-consensus network, it is presented in a form that is useful for the miners, with most of the calculation and hashing done, so that a miner needs to simply add the coinbase and finalise the state root calculation.
The reason pre-consensus network can be run more efficiently than the main network, is that it will operate in ReGenesis mode, having the active state in RAM, and therefore limiting the actual cost of state accessing operations. Nodes in the pre-consensus network will rely on the transaction pool network to append witnesses to the transactions and filter out most of the transactions that would fail due to the insufficient witness, or very large witness.
In order to assist transaction injecting nodes with performing their functions, there will be a regular creation and distribution of various snapshots via BitTorrent:
The content addresses of these files will be hard-coded in the implementation, because we can expect to make at least 1 code release during 1 ReGenesis cycle.
One of the main critiques of Stateless Ethereum project so far was the potentially large block witnesses. With ReGenesis, the expectation is that at least for some time after each ReGenesis event, the blocks will be large due to all transactions in them having to carry witnesses for the state they are accessing. In order to deal with this problem, we can introduce a concept of “thin blocks”. The idea is that instead of having every block body include the entire transaction with its witness, we will have block bodies include only the list of transaction IDs, and block witness (which is much smaller for ReGeneis, since it only need to ensure coinbase is in the active state). Transactions can be obtained via transaction pool, or if they are gone, full size blocks can be downloaded from peers on demand (but we assume that most peers who need to receive blocks fast, will be able to reconstruct them from the list of transaction IDs and block witnesses).
We can construct the rules of pre-consensus such that:
In order to check (2) we would need to use observations from the transaction pool network. With such rules, we can end up with the small set of candidate blocks such that we are ambivalent about which block should win in the pre-consensus, because the rules above ensure that they are all good. However, an adversary performing a Sybil attack, can make pre-consensus stall, and never make any decision. To overcome this issue, the “Snowglobe” proposal for Bitcoin Cash suggested using “coin age” to resist Sybil attacks: https://github.com/tyler-smith/snowglobe/blob/master/spec/snowglobe.md#sybil-resistance-via-coin-age
Coin age can be very easily observe in UTXO blockchains, but in Ethereum we might need to create a “coin age” contract to emulate that. In such contract, anyone can deposit some ETH, and register a public key that would be used for verifying handshake challenges when nodes join pre-consensus. Withdrawal can happen at any time, but it immediately invalidates the eligibility of the registered public key.
If pre-consensus is achieved for a block, and miner followed it, and produced the pre-agreed block with its coinbase and its mining nonce, then such block can be propagated to the peers using just 60 bytes of information (20 bytes for coinbase address, 8 bytes for the mining nonce, and 32 bytes for
extraData). This is almost as small as a block hash. Currently, go-ethereum uses this logic to propagate and announce new blocks. If it has
N peers, it will randomly select
sqrt(N) peers, and propagate the new block to them (with full body). For the rest of the peers, the new block is just announced (only hash of the block is sent). If the mined block followed pre-consensus, then such block can be propagated to all peers, and reach all corners of the network faster. This creates a potential incentive for miners to obey the pre-consensus.
So far these tasks are not arranged in a “topological” order, so it is mostly to give impression of may need to be done. This section is heavily biased to the turbo-geth and its architecture.
ethand transaction pool networks