In today’s Road to Neo3 episode, we’ll begin to look at another improvement under development for Neo3; determining global state by persisting a state root hash. Providing a globally agreed state makes it possible to solve trust issues with light wallets, ensures consistent storage information between nodes, and can result in performance gains achieved by separating block and state persistence.

Node state

The state of a node can be thought of as a snapshot of all stored values at a given time. For example, the state of a NEP-5 contract’s storage is predominantly composed of user token balances. The state of a node’s storage would in turn be a summary of all data stored in all contracts.

Currently on Neo2, nodes on the network do not have an agreed global state. Instead, each node calculates its state locally, occurring whenever a block is processed. This can introduce problems where even though all nodes on the network share the exact same block information (as validated by the consensus nodes), one node’s locally determined state could differ from another due to difference in interpretation of that block data.

This problem, which could stem from differences in VM version or node type, can be quickly identified if there is consensus on a persistent state root. Discussions into state persistence were first opened by Erik Zhang, who noted that performance improvements could arise if block and state persistence were decoupled:

“If we separate state persistence from block persistence, the consensus node can quickly process transactions without waiting for state writes and smart contract execution.”

Since state calculation is performed while blocks are being persisted, consensus nodes may be held up from assisting in the block creation process—ordering transactions, calculating fees, and participating in block validation. Due to this, the synchronization between state calculation and block persistence can cause delays in new blocks being relayed, potentially affecting the throughput of the network.

Light client trust problem

Preliminary discussions on this issue were focused on providing state persistence in a form that was decoupled from blocks, however an additional issue resulting from the lack of global state caused an alternate approach to be suggested. Due to their dependence on information retrieved from third-party nodes, light clients such as mobile wallets currently experience a trust dilemma.

Since light clients do not store the full blockchain with which they could verify the state of a user’s balance, they have no way to ensure that data retrieved from an API or RPC node is correct. The proposed solution to this depends on the implementation of a cryptographically authenticated data structure known as a Merkle Patricia trie (MPT).

MPT is a combination between a Patricia trie (a special version of a binary radix trie), used for efficient lookups, insertion, and deletion, and a Merkle tree, used to provide quick cryptographic verification of the whole data structure through a single “root” hash.

After MPT has been implemented, nodes on the network can continue to calculate a local state in a deterministic manner, updating the Merkle root as it goes. This root hash can then be validated along with the block during the consensus process, where it can then be persisted and referenced by other nodes to verify the integrity of data.

We’ll explore Merkle Patricia tries and the ongoing implementation for Neo3 in more detail in a later episode.

State root in block headers

To solve these trust issues and provide global state, the addition of state root to block headers was proposed. This would allow light clients to store only the block headers, using them for proof of state to check the authenticity of data provided by RPC nodes. Allowing verification in this manner solves the trust problem with light clients, allowing information to be confirmed without requiring a full copy of the blockchain.

Further, inclusion in a block header ensures that every node on the network is in agreement with the global state. This ensures that all nodes have the same view of contract storage, eliminating consistency issues by providing a method for nodes to check whether they are in alignment with consensus nodes.

Although a useful solution in its own right, this approach does come with some drawbacks. In the next article, we’ll follow developer discussions on this topic and highlight the unique bug-fixing capabilities of Neo that the core developers aim to preserve during state root implementation.

Tags: