Road to Neo3: Oracle transactions and the Broadcast Model

Picking up from where we left off in the Introduction to Oracles, in this article we’ll look at oracle transactions, the most fundamental component of the oracle solution, examining the proposed transaction structure and how these transactions are relayed through the network.

In order to make a data request to be validated by the oracle network, an oracle transaction must first be created by a contract or client. These special transactions will allow interaction with the oracle network by triggering a newly created oracle system call with its two required parameters: the destination URL and a filter for shortening any downloaded content.

For example, consider a smart contract that takes the form of a decentralized marketplace where users can buy and sell items to each other. If items were priced on the marketplace in GAS, a large shift in the market price of GAS could catch vendors by surprise and cause unfair trades to occur. Instead, the oracle network could be used to allow items to be priced in a fiat currency, using the oracle calls to monitor the market price of GAS so that the contract can calculate the appropriate GAS price dynamically.

In this example, the oracle call URL might be directed at a price monitoring API, such as CryptoCompare. Using the destination URL “https://min-api.cryptocompare.com/data/price?fsym=GAS&tsyms=USD,EUR,GBP” in the oracle call would cause each oracle node to retrieve a price conversion for GAS against three fiat currencies:

{"USD":1.068,"EUR":0.9615,"GBP":0.8217}

The filter could be used to restrict this data so that only a certain part of the response is returned for validation by oracles and eventual inclusion in the block. This helps optimize blockchain and network usage, and is a useful trait for achieving determinism in the event that APIs provide varied responses due to extra data that is unnecessary for the contract (e.g current time).

After downloading and filtering the response, oracle nodes can come to a majority agreement on the correct return value, which can be subsequently included in a block. This process allows a user or contract to use the oracle network to directly access any required information and bring it into the blockchain for further use.

Transaction structure

An oracle transaction has two differences from normal transactions on Neo3, which consist of signed and unsigned data. In a normal transaction, the signed part holds information such as the sender, version, and any additional attributes. The unsigned part consists of the transaction signature.

The proposed structure for an oracle transaction adds a new attribute called ExpectedResultHash to the attributes found within the signed portion of the transaction. Although it’s likely that most oracle requests will be made without the result being known beforehand, this inclusion is useful as it allows the user to specify a result to be checked in order for the transaction to be validated. As core developer Belane explains, this also provides a simple flag to be checked by a node, allowing it to easily determine whether a transaction is an oracle transaction or not.

In situations where the user does not expect any particular result, this hash will instead be zero-filled, causing the result to depend entirely on consensus between the oracle nodes.

The other difference is a new OracleAgreement section, located in the unsigned part and host to the DownloadCache, which contains three important pieces of information: the URL to retrieve data from, a filter such as JSONPath or XPath for restricting downloaded content to only the required information, and the content itself (following execution by oracle nodes).

Broadcast model

One notable advantage of the Neo oracle implementation is that it makes strong use of the single block transaction finality provided by dBFT to optimize confirmation times, network traffic, and chain storage space. This is due to oracle calls on the Neo network only requiring a single transaction to be processed and confirmed within a single block, making the result immediately available to contracts.

The proposed memory pool and broadcast model shows how oracle transactions are relayed through the network, a process which is assisted by all nodes on the network in addition to oracle nodes themselves. After an oracle transaction has been broadcast into the network, each node will store the currently unverified transaction in its memory pool as a known hash and pass it along to other nodes.

Oracle nodes that discover oracle transactions through this process will complete any included requests, using the URL and applying the filter as provided in the DownloadCache. These nodes will then reach consensus on a return value by appending results and signatures to the OracleAgreement section of the oracle transaction.

Once sufficient signatures are gathered to pass the approval threshold (currently TBD), the oracle transaction can be considered verified and is ready to be included in a block by consensus nodes. In effect, the same transaction is used for both oracle invocation and for finalizing the results, with previously off-chain data now stored on-chain where it can be used by contracts.

In the next article, we’ll look more in depth at how oracles calls are executed, how consensus on results is achieved, and provide more information on filters and supported protocols.