An architecture for optimal all-to-all personalized communication

Abstract

In all-to-all personalized communication (AAPC), every node of a parallel system sends a potentially unique packet to every other node. AAPC is an important primitive operation for modern parallel compilers, since it is used to redistribute data structures during parallel computations. As an extremely dense communication pattern, AAPC causes congestion in many types of networks and therefore executes very poorly on general purpose, asynchronous message passing routers.We present and evaluate a network architecture that executes allto-all communication optimally on a two-dimensional torus. The router combines optimal partitions of the AAPC step with a selfsynchronizing switching mechanism integrated into a conventional wormhole router. Optimality is achieved by routing along shortest paths while fully utilizing all links. A simple hardware addition for synchronized message switching can guarantee optimal AAPC routing in many existing network architectures.The flexible communication agent of the iWarp VLSI component allowed us to implement an efficient prototype for the evaluation of the hardware complexity as well as possible software overheads. The measured performance on an 8 8 torus exceeded 2 GigaBytes/sec or 80% of the limit set by the raw speed of the interconnects. We make a quantitative comparison of the AAPC router with a conventional message passing system. The potential gain of such a router for larger parallel programs is illustrated with the example of a two-dimensional Fast Fourier Transform.

A Message Scheduling Scheme for All-to-All Personalized Communication on Ethernet Switched Clusters

“…Similar to other AAPC scheduling schemes [6], our scheme partitions AAPC into contention-free phases and fully utilizes the bandwidth in the bottleneck links in all phases. Hence, realizing AAPC with the contention-free phases can potentially achieve the minimum communication completion time.…”

“…A large number of optimal message scheduling algorithms for different network topologies with different network models were developed. Many of the algorithms were designed for specific network topologies that are used in parallel machines, including hypercube [7,24], mesh [1,18,17,22], torus [6,11], k-ary n-cube [24], and fat tree [3,16]. Heuristic algorithms were developed for AAPC on irregular topologies [5,14].…”

Performance of Multi-hop Communications Using Logical Topologies on Optical Torus Networks

“…connections on ring and torus topologies can be found in [3]. Using an algorithm in [3], a multiplexing degree of…”

Modeling Parallel Bandwidth: Local versus Global Restrictions

“…Many previous works have studied the total-exchange (also called complete exchange and all-to-all personalized communication) primitive in which each processor has a distinct message to send to every other processor (see, e.g., [9], [14], [18], [31], [33], [34], [36], [41], [46] and [47]). The total-exchange primitive has been incorporated into communication libraries such as the Collective Communication Library provided with the IBM SP-2 [8].…”