On May 14th, NEO St. Petersburg Competence Center co-founder Sergei Liubich presented the R&D team’s flagship project, NeoFS, at Consensus 2019 in New York. In the presentation, Liubich noted the requirements and issues present in the data storage market and demonstrated the functionality of the NeoFS distributed decentralized storage platform.

NeoFS was introduced as a distributed peer-to-peer data storage service that provides users with full control over their data whilst avoiding single points of failure. Example use cases were also noted, including:

  • Private data storage for users
  • dApps and content delivery networks
  • Data exchange for SMEs (small to medium-sized enterprises)
  • Unstructured IoT (Internet of Things) data
  • Fixed document storage

By offering a wide scope of use cases including those offered by traditional data storage services, NEO SPCC hopes that NeoFS will provide a more cost-effective solution for users and businesses with all the benefits of fault-tolerant infrastructure.

NeoFS

NeoFS uses smart contracts on the NEO blockchain for the exchange of assets. This allows storage services to be purchased or rented out in a transparent manner, and also allows dApps on NEO to store their data with NeoFS for use in their own contracts.

The infrastructure of NeoFS involves the use of two node rings; inner and outer nodes. The outer nodes are used for storage and may take the form of storage clusters, servers, or personal computers. The inner ring is responsible for data auditing and maintenance.

The architecture and novel data auditing methods have been developed for NeoFS with the goal of guaranteeing data integrity, placement, and availability, which would ensure reliable and trustless user experience.

More information on the novel approaches used may be found in the Research Plan.

Video Demo

To accompany the presentation, NEO SPCC provided a video demo to show how data is stored across the network according to the user-defined placement rule, how payments can be handled using the NEO blockchain, and how data can then be audited in a scalable manner.

The video begins by demonstrating the storage of data collected from the network in real-time. First, a container is created via console with a storage policy that requires data to be stored in three countries.

Data is placed according to the storage policy.

These policies are flexible and can be used to select specific countries in which to store data and whether it is stored on HDD or SSD devices. In the demo, a text file is uploaded and stored in three locations; North America, Europe, and Asia.

The demo then showcases the use of a NEO wallet that is connected to NeoFS. GAS is deposited to the NeoFS contract, where it can then be used to purchase data storage services. New containers can then be created with their own storage policies, which can then be used to update new files such as videos or images.

GAS payment integration is demonstrated.

Liubich demonstrates with a small video file, which is uploaded in data chunks that are split across nodes in the system in a balanced manner. An additional image file is uploaded, and then the objects are retrieved from the system. Finally, the video file is deleted from the container, resulting in the related chunks being removed from the network.

To conclude the demo, the auditing process is outlined. Liubich notes that any number of storage groups can be created under a container, and all files in these groups are audited at once. As the results of the audits are used to determine payments, this prevents nodes from deleting individual files to game the system.

Corrupted chunks are detected in the audit.

A successful data audit is run to demonstrate that the storage groups are passing validation, and then the data is intentionally corrupted. During the next audit, the provided histogram updates to display audit failure. The nodes that have failed the audit due to this corrupted data would not receive payment. To restore the storage policy, the remaining nodes then replicate all corrupted chunks, ensuring the integrity and redundancy of data.

As the storage node configuration can change when the network map updates, a migration process is included that allows data to be transferred to new nodes. The majority of the network is unchanged to preserve scalability, and only containers with new nodes are migrated. Liubich demonstrates this migration by adding a new node to the network, with a percentage of chunks being migrated proportionally based on the size of the network.

The full video demo can be found below:

In its Q1 development report, NEO SPCC was reported to be on target for its goal of a proof-of-concept launch in Q3 2019, a target that was reiterated by Liubich in the conclusion of the presentation.

Tags: ,