High-bandwidth data dissemination for large-scale distributed systems

Abstract

IBMThis article focuses on the multireceiver data dissemination problem. Initially, IP multicast formed the basis for efficiently supporting such distribution. More recently, overlay networks have emerged to support point-to-multipoint communication. Both techniques focus on constructing trees rooted at the source to distribute content among all interested receivers. We argue, however, that trees have two fundamental limitations for data dissemination. First, since all data comes from a single parent, participants must often continuously probe in search of a parent with an acceptable level of bandwidth. Second, due to packet losses and failures, available bandwidth is monotonically decreasing down the tree.To address these limitations, we present Bullet, a data dissemination mesh that takes advantage of the computational and storage capabilities of end hosts to create a distribution structure where a node receives data in parallel from multiple peers. For the mesh to deliver improved bandwidth and reliability, we need to solve several key problems: (i) disseminating disjoint data over the mesh, (ii) locating missing content, (iii) finding who to peer with (peering strategy), (iv) Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or direct commercial advantage and that copies show this notice on the first page or initial screen of a display along with the full citation. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this work in other works requires prior specific permission and/or a fee. Permissions may be requested from Publications Dept., ACM, Inc., 2 Penn Plaza, Suite 701, New York, NY 10121-0701 USA, fax +1 ( right rate from all peers (flow control), and (v) recovering from failures and adapting to dynamically changing network conditions. Additionally, the system should be self-adjusting and should have few user-adjustable parameter settings. We describe our approach to addressing all of these problems in a working, deployed system across the Internet. Bullet outperforms state-of-the-art systems, including BitTorrent, by 25-70% and exhibits strong performance and reliability in a range of deployment settings. In addition, we find that, relative to tree-based solutions, Bullet reduces the need to perform expensive bandwidth probing.

Achieving Bounded Matching Delay and Maximized Throughput in Information Dissemination Management

“…For example, Bullet, a multi-receiver data dissemination mesh that aims for the high-bandwidth data dissemination for large-scale distributed systems, has been presented by Kostić et al [9]. Voulgaris et al propose a self-organizing content-based publisher/subscriber system for dynamic largescale collaborative networks [10].…”

Removing the blinders: Using information to mitigate adversaries in adaptive overlays

“…One important facet of the solution is that it uses information already present in the system, avoiding extra link stress. It should also be noted that many adaptive protocols utilize similar metrics for adaptation [4]- [7], [15], [19]- [21], allowing our solution to be readily applied.…”

“…To show the general applicability of our defense technique as discussed in Section IV, we apply it to BulletPrime [15]. BulletPrime is a mesh-based P2P file sharing application similar to BitTorrent.…”

“…• We demonstrate that our solution is a general mechanism that can be readily applied to a variety of protocols without the creation of a formal specification and minimal change to the protocol by using it to add robustness to BulletPrime [15], which uses a mesh-based overlay.…”

A synchronization protocol for supporting peer-to-peer multiplayer online games in overlay networks

“…It is not the aim of this paper to propose a specific scheme to build such overlay. Many solutions exist to structure a P2P overlay network, such as those presented in [20,32,33,36,37,39]. For the sake of a simpler discussion of our scheme, we assume that the overlay is structured as a tree.…”