Ethereum 1x Definition (part 3)

Methodology (continuation from parts 1 and 2)

In part 1, we define agents, their contributions to the Ethereum system, and their challenges. This part deals with causes for these challenges, and the solution that have been proposed for these causes.

Causes

We try to understand the main causes of each challenge. Where it adds to understanding, we talk about sub-causes







causes



cause

cause



sub_cause

sub-cause



cause->sub_cause





challenge

challenge



challenge->cause





Solutions

Proposed solutions should be targeted at the causes but may have side-effects (some solutions may be causes for other challenges, though perhaps less critical than the ones they are trying to address). These side effects are shown as dotted lines.







solutions



solution

solution



cause

cause



cause->solution





challenge

challenge



challenge->solution





Summary of causes

Here we summarise the causes for the challenges. Most of them are technological, though some could be viewed as organisational.

  1. Large (and growing) state
  2. Snapshot sync algorithm implementation deficiencies
  3. Increasingly intricate data management that an Ethereum node needs to perform
  4. Functional coupling on implementation level
  5. Functional coupling on protocol level
  6. Large semantic gap between EVM and many dapps
  7. EVM design favours expressiveness over tractability
  8. Transaction fee market behaves sub-optimally when blocks are at the block gas limit (which is currently almost always).
  9. Possibility of long chain re-orgs.
  10. Over-reliance of spontaneous contributions.

Description of causes

Large (and growing) state

Ethereum’s state is a data structure that needs to be implicitly constructed, stored and accessed in order to execute arbitrary transactions. This is because a transaction may theoretically access any item in the current state. The state size grew beyond the capacity of RAM (Random Access Memory) on average computers some time in 2017. After that point, RAM could only cache certain portions of the state, whereas the entire state resides on persistent storage devices. Any caching strategy apart from keeping a random portion of the accessible state, would be vulnerable to attack. Therefore, assuming that random caching strategy, the cache hit ratio would very close to the ratio of size of the cache to the size of the entire state. And cache miss would mean accessing devices with much higher latency.
Because of the further growth of the state, it became impractical to use HDD (Hard Disk Drives - storage devices with mechanically spinning disks) for storage of the state, due to the high latency of the access. Even SDD (Solid State Drives) are on the edge of being appropriate. Devices such as NVM (Non Volatile Memory) are now required to ensure good performance. However, such devices are still relatively expensive and their price proportional to capacity ($/Gb) is highly non linear after a certain point.
State size also places a significant burden on the new participants in the Ethereum network. Most popular way of joining the network at this moment is so-called “snapshot synchronisation”. It is the process in which the new-joiner downloads the entire state from the existing peers. The sheer size of the state puts a high demand on the bandwidth quality. Dealing with the network latencies requires sophisticated algorithms for downloading. And the ever-changing nature of the state (it keeps changing during the download) either puts snapshot sync at odds with state history pruning, or requires even more sophisticated algorithms for the downloading of the state.

Snapshot sync algorithm implementation deficiencies

Snapshot synchronisation (briefly described in the second ) can be implemented in a variety of ways. The dominant algorithm used at the moment is called “fast sync”. It has been designed in the circumstances when the state size was still reasonably small. The main advantage of this algorithm is its simplicity. However, implementations may also vary in efficiency. For example, they might be sensitive to network latencies (how much slower algorithm performs with an increase in latency), or require a lot of overhead traffic (how much more data algorithm transmits with an increase of state size).

Increasingly intricate data management that an Ethereum node needs to perform

The hardest part of a main-net capable implementation seems to be the data management. Ethereum node needs to transmit, process and store large amounts of data. To do so efficiently, it needs non-trivial techniques, like data caching strategies, partitioning data by usage type. Such techniques, however, are currently not considered in Ethereum specification documents or other literature.

Functional coupling on the implementation level

Example - persistent storage of state as a Particia tree, which couples logical structure needed to compute state root, and physical structure needed to store the state. Such coupling makes it harder to switch to binary Merkle tree for hashing the state.

Functional coupling on protocol level

In the design of Ethereum, there some crucial concepts with double or triple functions. First example: gas is used as a charge for resources, as well as means of restricting callbacks and recursion depth. Second example: nonce of accounts is used for both replay protection and as input for generating of the contract addresses. Third example (though this is a prevalent implementation choice rather than a requirement of the specification): Particia Merkle tree is used for both defining what the state root hash is, and for storing data. Functional coupling makes the design inflexible and causes issues when something about the concept needs to be changed. For example, changing gas cost of some operations, like reduction for SSTORE and increase for SLOAD clashes with the concept of call stipend, which is part of the restricting function of gas.

Large semantic gap between EVM and many dapps

Definition of semantic gap from www.techopedia.com:

The “semantic gap” as it is often referenced in IT is the difference between high-level programming sets in various computer languages, and the simple computing instructions that microprocessors work with in machine language. This classic difference has compelled engineers and designers to look at different ways of mediating between high-level language and basic machine language.

In the past, engineers tried to bridge the semantic gap by making microprocessors more complex, as with Complex Instruction Set Computing (CISC) models. However, they found that it could be just as effective, if not more so, to design Reduced Instruction Set Computing (RISC) models. The philosophy is that microprocessors do not have to do complex work, but can break the high-level instructions down into simple steps. That resonates with how semantic programming is compiled or broken down into machine language. The semantic gap illustrates the difference between humans and computers and how they process data.

After EVM has been designed and implemented, it turned out that there are many use cases, specifically involving cryptographic primitives (e.g. hashing algorithms, digital signatures) whose implementation in EVM opcodes would consume prohibitive amounts of gas.

EVM design favours expressiveness over tractability

A virtual machine

Over-reliance on spontaneous contributions

By “spontaneous voluntary contributions” we understand contributions to core implementation by people who are not explicitly asked to do the work, but do it, because they found it interesting and/or important.
By “managed development” we understand development in the core implementations that are directed by some leadership, according to some implementation plan.
Both ways of development have their pros and cons. It seems that in the current circumstances we mainly rely on the spontaneous voluntary contributions and that seems to leave important gaps and technical debt.

Summary of solutions

At this moment, only solutions to the technological causes are listed and explained.

  1. State rent
  2. Stateless clients
  3. Advanced sync protocols and algorithms (Leaf sync, Beam sync, Firehose/Red Queen)
  4. Content Distribution Networks (Cloudflare, Swarm, etc.) to store blocks and/or state
  5. Decoupling of two functions of gas
  6. Burning part of the transaction fees (EIP-1559)
  7. Alternative data layouts and databases
  8. Adding pre-compiles (generic elliptic curve pre-compiles, BLAKE hash function)
  9. Introducing WASM engine
  10. Finality gadget to link Ethereum 1x to the Beacon chain
  11. Upgrade of the Proof Of Work algorithm (ProgPOW)
  12. State management component (interface design)

Description of solutions

State rent

State rent is the only currently proposed solution to decreasing of the state size. All other solutions are concerned with how to deal with the large (and growing) state size more efficiently and prevent some of the most obvious failure modes.

Stateless clients

Stateless clients is one approach to improve performance of Ethereum client implementations while processing blocks of transactions. Specifically, it seeks to ease the increasing burden that is the state size. It does so by removing the need to download, or implicitly construct and maintain the state, for most of the participants in the Ethereum network. The requirement to access the state is removed by introducing another type of data packets (existing data packet types are, for example, blocks and transactions) to be gossipped around the p2p network. We call this data packets “block witnesses”. For each block we have one corresponding block witness. The two main properties that block witnesses have:

  1. It is possible verify efficiently that the block witness is indeed constructed from the correct version of the Ethereum state.
  2. Block witness has all the required information to make it possible to execute the corresponding block.

More details can be found here: https://medium.com/@akhounov/data-from-the-ethereum-stateless-prototype-8c69479c8abc