Genet: A Quickly Scalable Fat-Tree Overlay for Personal Volunteer Computing using WebRTC

Abstract

WebRTC enables browsers to exchange data directly but the number of possible concurrent connections to a single source is limited. We overcome the limitation by organizing participants in a fat-tree overlay: when the maximum number of connections of a tree node is reached, the new participants connect to the node's children. Our design quickly scales when a large number of participants join in a short amount of time, by relying on a novel scheme that only requires local information to route connection messages: the destination is derived from the hash value of the combined identifiers of the message's source and of the node that is holding the message. The scheme provides deterministic routing of a sequence of connection messages from a single source and probabilistic balancing of newer connections among the leaves. We show that this design puts at least 83% of nodes at the same depth as a deterministic algorithm, can connect a thousand browser windows in 21-55 seconds in a local network, and can be deployed for volunteer computing to tap into 320 cores in less than 30 seconds on a local network to increase the total throughput on the Collatz application by two orders of magnitude compared to a single core.

You Don't Need a Ledger

“…In the current network, peers connect to other peers randomly. By moving to a structured P2P network, such as a fat-tree overlay [10], the replication can flow through the network more efficient. At last, the consensus protocol is now used for each token individually.…”

A Volunteer Computing Architecture for Computational Workflows on Decentralized Web

“…The task deployment is based on a Node.js master server (Stream Lender) responsible for scheduling functions and collecting their results. Genet [35] is an evolution of Pando that tries to overcome its scalability problems -due to direct connections handled with WebRTC -by using a fat-tree overlay network (where processors are located on the leaves and internal nodes relay data for all their children). Genet differs from Pando in managing browser connections, switching a node's role from management to relay when its direct connections (children) reach a given threshold.…”