Efficient All-to-All Communication Patterns in Hypercube and Mesh Topologies

Scott, David

doi:10.1109/dmcc.1991.633174

Cited by 100 publications

(53 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, although there is an exponential number of partitions of an integer d, we shall prove that the number of faces on the hull of optimality is O(vQ). 15 …”

mentioning

confidence: 99%

Multiphase complete exchange: a theoretical analysis

Bokhari

1996

IEEE Trans. Comput.

View full text Add to dashboard Cite

Complete Exchange requires each of N processors to send a unique message to each of the remaining N -1 processors. For a circuit switched hypercube with N = 2d processors, the Direct and Standard algorithms for Complete Exchange are optimal for very large and very small message sizes, respectively. For intermediate sizes, a hybrid Multiphase algorithm is better. This carries out Direct exchanges on a set of subcubes whose dimensions are a partition of the intege-d. The best such algorithm for a given message size 7n could hitherto only be found by enumerating all partitions of d.The Multiphase algorithm is analyzed assuming a high performance communication network. It is proved that only algorithms corresponding to equipartitions of d (partitions in which the maximum and minimum elements differ by at most 1) can possibly be optimal.The run times of these algorithms plotted against m form a hull of optimality. It is proved that, although there is an exponential nur her of partitions, (1) the number of faces on this hall is O(v'T), (2) the hull can be found in O(v/d) time, and (3) once it has been found, the optimal algorithm for any given in can be found in O(log d) time.These results provide a very fast technique for minimizing communication overhead in many important applications, such as matrix transpose, Fast Fourier transform and ADI.

show abstract

“…Therefore, although there is an exponential number of partitions of an integer d, we shall prove that the number of faces on the hull of optimality is O(vQ). 15 …”

mentioning

confidence: 99%

Multiphase complete exchange: a theoretical analysis

Bokhari

1996

IEEE Trans. Comput.

View full text Add to dashboard Cite

show abstract

“…Many all-to-all broadcast algorithms were designed for specific network topologies that are used in parallel machines, including hypercube [8,20], mesh [15,18,21], torus [21], k-ary n-cube [20], fat tree [10], and star [14]. Work in [9] optimizes MPI collective communications, including M P I Allgather, on wide area networks.…”

Section: Related Workmentioning

confidence: 99%

Bandwidth Efficient All-to-All Broadcast on Switched Clusters

Faraj¹,

Patarasuk²,

Zhong³

2005

2005 IEEE International Conference on Cluster Computing

View full text Add to dashboard Cite

We develop an all-to-all broadcast scheme that achieves maximum bandwidth efficiency for clusters with tree topologies. Using our scheme for clusters with cut-through switches, any tree topology can support allto-all broadcast as efficiently as a single switch connecting all machines when the message size is sufficiently large. Since a tree topology can be embedded in almost any connected network, it follows that efficient all-toall broadcast can be achieved in almost all topologies, regular or irregular. To perform all-to-all broadcast efficiently on clusters with store-and-forward switches, the algorithm must minimize the communication path lengths in addition to maximizing bandwidth efficiency. This turns out to be a harder algorithmic problem. We develop schemes that give solutions to common cases for such systems. The performance of our algorithms is evaluated on Ethernet switched clusters with different topologies. The results confirm our theoretical finding. Furthermore, depending on the topology, our algorithms sometimes out-perform the topology-unaware algorithms used in MPI libraries, including MPICH and LAM/MPI, to a very large degree.

show abstract

“…All-to-all personalized exchange occurs in many important parallel computing/networking applications, such as matrix transposition and fast Fourier transform (FFT) [1]- [2]. There has been much work for all-to-all personalized exchange in various networks; see, for example [3]- [9].…”

Section: Introductionmentioning

confidence: 99%

Optimal all-to-all personalized exchange in a class of optical multistage networks

Yang

Wang²

2001

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

All-to-all personalized exchange is one of the most dense collective communication patterns and occurs in many important parallel computing/networking applications. In this paper, we look into the issue of realizing all-to-all personalized exchange in optical multistage networks. Advances in electro-optic technologies have made optical communication a promising networking choice to meet the increasing demands for high channel bandwidth and low communication latency of high-performance computing/communication applications. Although optical multistage networks hold great promise and have demonstrated advantages over their electronic counterpart, they also hold their own challenges. Due to the unique properties of optics, crosstalk in optical switches should be avoided to make them work properly. In this paper, we will provide an optimal scheme for realizing all-to-all personalized exchange in a class of unique-path, selfrouting optical multistage networks crosstalk-free. The basic idea of realizing all-to-all personalized exchange in such a multistage network is to transform it to multiple semi-permutations, each of which can be realized crosstalk-free in a single pass, and take advantage of pipelined message transmission in consecutive passes. As can be seen, the time complexity of our all-to-all personalized exchange algorithms matches the lower bound of the communication delay in this type of network.

show abstract

Efficient All-to-All Communication Patterns in Hypercube and Mesh Topologies

Cited by 100 publications

References 1 publication

Multiphase complete exchange: a theoretical analysis

Multiphase complete exchange: a theoretical analysis

Bandwidth Efficient All-to-All Broadcast on Switched Clusters

Optimal all-to-all personalized exchange in a class of optical multistage networks

Contact Info

Product

Resources

About