An Approximation Algorithm for Broadcast Scheduling in Heterogeneous Clusters

Liu, Pangfeng; Wang, Dawei; Guo, Yi-Heng

doi:10.1007/978-3-540-24686-2_3

Cited by 3 publications

(3 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…as Banikazemi [24], Bhat [4], Liu [5], Park [25], Mateescu [26] and Vorakosit [27] try to generate optimal broadcast trees according to a given root process.…”

Section: Grid-aware Communication Schedulingmentioning

confidence: 99%

A framework for adaptive collective communications for heterogeneous hierarchical computing systems

Steffenel

Mounié

2008

Journal of Computer and System Sciences

View full text Add to dashboard Cite

Collective communication operations are widely used in MPI applications and play an important role in their performance. However, the network heterogeneity inherent to grid environments represent a great challenge to develop efficient high performance computing applications. In this work we propose a generic framework based on communication models and adaptive techniques for dealing with collective communication patterns on grid platforms. Toward this goal, we address the hierarchical organization of the grid, selecting the most efficient communication algorithms at each network level. Our framework is also adaptive to grid load dynamics since it considers transient network characteristics for dividing the nodes into clusters. Our experiments with the broadcast operation on a real-grid setup indicate that an adaptive framework allows significant performance improvements on MPI collective communications.

show abstract

“…as Banikazemi [24], Bhat [4], Liu [5], Park [25], Mateescu [26] and Vorakosit [27] try to generate optimal broadcast trees according to a given root process.…”

Section: Grid-aware Communication Schedulingmentioning

confidence: 99%

A framework for adaptive collective communications for heterogeneous hierarchical computing systems

Steffenel

Mounié

2008

Journal of Computer and System Sciences

View full text Add to dashboard Cite

show abstract

“…It is very intuitive that one might want to use a heterogeneous broadcast algorithm [6,7,10,22,25,26] and work in the reverse direction for reduction, since if all the data flow in the opposite direction, a broadcast becomes a reduction. However, this is not the case because in a reduction every processor must send and contribute its data, so every processor must send at least one message.…”

Section: Introductionmentioning

confidence: 99%

An Approximation Algorithm and Dynamic Programming for Reduction in Heterogeneous Environments

2007

Self Cite

View full text Add to dashboard Cite

Network of workstation (NOW) is a cost-effective alternative to massively parallel supercomputers. As commercially available off-the-shelf processors become cheaper and faster, it is now possible to build a cluster that provides high computing power within a limited budget. However, a cluster may consist of different types of processors and this heterogeneity complicates the design of efficient collective communication protocols. For example, it is a very hard combinatorial problem to find an optimal reduction schedule for such heterogeneous clusters. Nevertheless, we show that a simple technique called slowest-node-first (SNF) is very effective in designing efficient reduction protocols for heterogeneous clusters. First, we show that SNF is actually a 2-approximation algorithm, which means that an SNF schedule length is always within twice of the optimal schedule length, no matter what kind of cluster is given. In addition, we show that SNF does give the optimal reduction time when the cluster consists of two types of processors, when the ratio of communication speed between them is at least two. When the communication speed ratio is less than two, we develop a dynamic programming technique to find the optimal schedule. Our dynamic programming utilizes the monotone property of the objective function, and can significantly reduce the amount of computation time. Finally, combined with an approximation algorithm for broadcast 2004, we propose an all-reduction algorithm which sends the reduction answer to all processors, with approximation ratio 3.5. We conduct three groups of experiments. First, we show that SNF performs better than the built-in MPI_Reduce in a test cluster. Second, we observe a factor of 93 times saving in computation time to find the optimal schedule, when compared with a naive dynamic programming implementation. Thirdly, we apply the theoretical results to a branch-and-bound search and show that they can reduce the search time of the optimal reduction schedule by a factor of 500, when the cluster has three kinds of processors.

show abstract

“…While some works just focus on the search for the best broadcast tree of a network [5], most authors such as Banikazemi [1], Bhat [6], Liu [13], Park [17], Mateescu [16] and Vorakosit [18] try to generate optimal broadcast trees according to a given root process, which corresponds to the MPI_Bcast operation.…”

Section: Related Workmentioning

confidence: 99%