On using connection-oriented vs. connection-less transport for performance and scalability of collective and one-sided operations

Mamidala, Amith R.; Narravula, S.; Vishnu, Abhinav; Santhanaraman, Gopalakrishnan; Panda, Dhabaleswar K.

doi:10.1145/1229428.1229437

Cited by 6 publications

(1 citation statement)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Researchers have previously shown the benefit of using hardware-based multicast over UD to increase collective performance in MVAPICH [18,20]. Mamidala, et al, has also shown the benefit of using UD for MPI Alltoall operations [21]. Our work instead focuses on a complete MPI stack over UD for high-performance and scalable point-to-point performance.…”

Section: Related Workmentioning

confidence: 99%

High performance MPI design using unreliable datagram for ultra-scale InfiniBand clusters

Koop

Sur

Gao

et al. 2007

Proceedings of the 21st Annual International Conference on Supercomputing

Self Cite

View full text Add to dashboard Cite

High-performance clusters have been growing rapidly in scale. Most of these clusters deploy a high-speed interconnect, such as InfiniBand, to achieve higher performance. Most scientific applications executing on these clusters use the Message Passing Interface (MPI) as the parallel programming model. Thus, the MPI library has a key role in achieving application performance by consuming as few resources as possible and enabling scalable performance. Stateof-the-art MPI implementations over InfiniBand primarily use the Reliable Connection (RC) transport due to its good performance and attractive features. However, the RC transport requires a connection between every pair of communicating processes, with each requiring several KB of memory. As clusters continue to scale, memory requirements in RC-based implementations increase. The connection-less Unreliable Datagram (UD) transport is an attractive alternative, which eliminates the need to dedicate memory for each pair of processes.In this paper we present a high-performance UD-based MPI design. We implement our design and compare the performance and resource usage with the RC-based MVAPICH. We evaluate NPB, SMG2000, Sweep3D, and sPPM up to 4K processes on an 9216-core InfiniBand cluster. For SMG2000, our prototype shows a 60% speedup and seven-fold reduction in memory for 4K processes. Additionally, based on our model, our design has an estimated 30 times reduction in memory over MVAPICH at 16K processes when all connections are created. To the best of our knowledge, this is the first research work that presents a high-performance MPI design over InfiniBand that is completely based on UD and can achieve near identical or better application performance than RC.

show abstract

Section: Related Workmentioning

confidence: 99%