Balanced Multicasting: High-throughput Communication for Grid Applications

Burger, Mathijs den; Kielmann, Thilo; Bal, Henri E.

doi:10.1109/sc.2005.15

Cited by 32 publications

(31 citation statements)

References 27 publications

(28 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In previous work [7], we have presented Balanced Multicasting, improving over FPFR by also taking bandwidth capacity into account. An example is shown in Fig.…”

Section: Balanced Multicastingmentioning

confidence: 99%

See 1 more Smart Citation

Collective Receiver-Initiated Multicast for Grid Applications

Burger

Kielmann

2011

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

Abstract-Grid applications often need to distribute large amounts of data efficiently from one cluster to multiple others (multicast). Existing sender-initiated methods arrange nodes in optimized tree structures, based on external network monitoring data. This dependence on monitoring data severely impacts both ease of deployment and adaptivity to dynamically changing network conditions. In this paper we present Robber, a collective, receiver-initiated, high-throughput multicast approach inspired by the BitTorrent protocol. Unlike BitTorrent, Robber is specifically designed to maximize the throughput between multiple cluster computers. Nodes in the same cluster work together as a collective that tries to steal data from peer clusters. Instead of using potentially outdated monitoring data, Robber automatically adapts to the currently achievable bandwidth ratios. Within a collective, nodes automatically tune the amount of data they steal remotely to their relative performance. Our experimental evaluation compares Robber to BitTorrent, to Balanced Multicasting, and to its predecessor MOB. Balanced Multicasting optimizes multicast trees based on external monitoring data, while MOB uses collective, receiver-initiated multicast with static load balancing. We show that both Robber and MOB outperform BitTorrent. They are competitive with Balanced Multicasting as long as the network bandwidth remains stable, and outperform it by wide margins when bandwidth changes dynamically. In large environments and heterogeneous clusters, Robber outperforms MOB.

show abstract

“…In previous work [7], we have presented Balanced Multicasting, improving over FPFR by also taking bandwidth capacity into account. An example is shown in Fig.…”

Section: Balanced Multicastingmentioning

confidence: 99%

“…(3) Network monitoring systems monitor the network itself; it remains a hard problem to translate this data (e.g., available bandwidth) to information that is meaningful to an application or multicasting algorithm (e.g., achievable bandwidth [6]). Our previous work on Balanced Multicasting [7] belongs to this category.…”

Section: Introductionmentioning

confidence: 99%

Collective Receiver-Initiated Multicast for Grid Applications

Burger

Kielmann

2011

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

show abstract

“…Because IP multicast has not been widely deployed, several application-level multicast schemes have been proposed for different applications [1,2,3,7,8,9,10]. Some of them are based on a single multicast tree [2,3,8] without considering the collaboration among peers to maximize the throughput, while others split the data over multiple trees to increase the overall throughput.…”

Section: Related Workmentioning

confidence: 99%

“…Both systems fail to utilize the full upload capacity of all the participating nodes in the multicast group, limiting the maximum overall throughput. In Balanced Multicasting [10], the authors propose a balancing of the maximum amount of upload capacity and achievable capacity in all nodes and paths of a network. Although Balanced Multicasting increases the throughput for grid applications, it provides limited scalability and adaptability.…”

Section: Related Workmentioning

confidence: 99%

Proximity-Aware Collaborative Multicast for Small P2P Communities

López-Fuentes¹,

Steinbach²

2007

2007 IEEE International Parallel and Distributed Processing Symposium

View full text Add to dashboard Cite

show abstract

“…It is very intuitive that one might want to use a heterogeneous broadcast algorithm [6,7,10,22,25,26] and work in the reverse direction for reduction, since if all the data flow in the opposite direction, a broadcast becomes a reduction. However, this is not the case because in a reduction every processor must send and contribute its data, so every processor must send at least one message.…”

Section: Introductionmentioning

confidence: 99%

An Approximation Algorithm and Dynamic Programming for Reduction in Heterogeneous Environments

2007

View full text Add to dashboard Cite

Network of workstation (NOW) is a cost-effective alternative to massively parallel supercomputers. As commercially available off-the-shelf processors become cheaper and faster, it is now possible to build a cluster that provides high computing power within a limited budget. However, a cluster may consist of different types of processors and this heterogeneity complicates the design of efficient collective communication protocols. For example, it is a very hard combinatorial problem to find an optimal reduction schedule for such heterogeneous clusters. Nevertheless, we show that a simple technique called slowest-node-first (SNF) is very effective in designing efficient reduction protocols for heterogeneous clusters. First, we show that SNF is actually a 2-approximation algorithm, which means that an SNF schedule length is always within twice of the optimal schedule length, no matter what kind of cluster is given. In addition, we show that SNF does give the optimal reduction time when the cluster consists of two types of processors, when the ratio of communication speed between them is at least two. When the communication speed ratio is less than two, we develop a dynamic programming technique to find the optimal schedule. Our dynamic programming utilizes the monotone property of the objective function, and can significantly reduce the amount of computation time. Finally, combined with an approximation algorithm for broadcast 2004, we propose an all-reduction algorithm which sends the reduction answer to all processors, with approximation ratio 3.5. We conduct three groups of experiments. First, we show that SNF performs better than the built-in MPI_Reduce in a test cluster. Second, we observe a factor of 93 times saving in computation time to find the optimal schedule, when compared with a naive dynamic programming implementation. Thirdly, we apply the theoretical results to a branch-and-bound search and show that they can reduce the search time of the optimal reduction schedule by a factor of 500, when the cluster has three kinds of processors.

show abstract

Balanced Multicasting: High-throughput Communication for Grid Applications

Cited by 32 publications

References 27 publications

Collective Receiver-Initiated Multicast for Grid Applications

Collective Receiver-Initiated Multicast for Grid Applications

Proximity-Aware Collaborative Multicast for Small P2P Communities

An Approximation Algorithm and Dynamic Programming for Reduction in Heterogeneous Environments

Contact Info

Product

Resources

About