Filecoin Retrieval Market: Community members sharing knowledge and demoing their latest project development

Overview

This April, Juan Benet, founder of Protocol Labs, hosted a Summit for individuals and companies, such as Nebula AI, who are working on projects in a Filecoin Retrieval Market.

The goal of this Summit is to create a loosely working group of community members working on Retrieval Market projects. These members will be collaborating to share information and provide project support. Project support can come in the form of design review, funding, constructive feedback, and dev support.

What is the Retrieval Market?

In Filecoin’s white papers, it describes two markets: the Storage Market and the Retrieval Market. In the Storage Market, miners are competing to provide reliable storage to clients at the lowest cost. The data stored between miner and client typically remain private. This is not the case in the Retrieval Market where content is meant to be accessed by the public. Miners of the Retrieval market compete to provide users with content at the lowest cost, but the highest reliability.

The Goal

Ultimate this group will be working to build the world’s best Content Delivery Network (CDN) for the Filecoin Retrieval Market. The Retrieval Market aims to be a worldwide, fast, and efficient distribution system to make IPFS content readily available. However, the goal of 2021 is a little more feasible: to build the first production version of the Retrieval Market.

2021 Retrieval Market goals outlined in Benet’s presentation:

• Network of payment channels
• 10,000+ nodes/devices
• 1,000+ People involved in retrieval mining
• Sub-second delivery of CIDs world-wide
• Beginnings of a network that can respond to shifting demands of content

What is Needed to Make the Retrieval Market Successful?

There are several requirements Benet mentions to make a CDN that can support the world’s activities. Latency will eventually be between 1 to 10 milliseconds. More realistically for 2021, the CDN should expect to have a latency rate of 100 milliseconds. For storage and retrieval, the number of objects the CDN would have to handle is 10¹⁸ objects. For this Retrieval Market, most of those objects will have random access. Finally, the number of retrieval miners, in the beginning, is expected to be in the low thousands. As the Retrieval Market grows, the bulk of the work should be done by upwards of 100, 000 miners.

Another factor to consider is location. In a CDN, the closer content is stored to a user, the faster the user can access the content. Therefore, it is important to build a network of global nodes and miners. The hope is that eventually groups of devices that will form a network of regions across the world. Nodes and miners worldwide will help information to be stored as close to the user as possible.

Benet also dives into the importance of the experience for both users and developers. Ease of use will be an important factor to the CDN to appeal to first-time users and developers. A great example Benet gives is Tik Tok; a CDN with global content, but has an extremely smooth user experience. The bulk of miners are expected to be individuals with small mining rigs. So, the CDN will need to be easy for these small miners to use.

Lastly, the Retrieval Market will be conducting operations, such as registering parties, reward parties, and creating mechanisms, off-chain. Without the need to validate each transaction with the blockchain, off-chain transactions can happen instantaneously with little to no transaction fees. Off-chain also allows users to conduct their transactions with more privacy and anonymity. However, the details of this off-chain transaction system have yet to be finalized.

How Nebula AI Fits into the Picture

During the Summit, Charles Cao, Nebula AI CEO, gave a demo on our latest project, a platform called FilSwan that helps clients find a suitable miner to store their data. Clients will post tasks to store their data and miners will submit a bid saying they are willing to store that data for X price. Clients will then be able to view the miner’s price range, sending time, retrieval time, and storage location. With this information, clients can pick the best miner for their task. Designed to be user-friendly, the platform is also compatible with multiple different cryptocurrency payments. Now, this project is nowhere near finished. More features are continuously being added.

To view the FilSwan Demo: https://www.youtube.com/watch?v=K8tNfsWSS_8&list=PL_0VrY55uV19TsJ8r2qsMyG2v2LIKEmv2&index=2

To view the 1st video in the Summit’s series: https://www.youtube.com/watch?v=QuE_NotFCXs&list=PL_0VrY55uV19TsJ8r2qsMyG2v2LIKEmv2&index=1

To view more about Filecoin’s Retrieval Market: https://www.youtube.com/watch?v=56HrKnlPiDs