Adaptive Recursive Doubling Algorithm for Collective Communication

Arap, Omer; Swany, Martin; Brown, Geoffrey; Himebaugh, Bryce

doi:10.1109/ipdpsw.2015.82

Cited by 7 publications

(1 citation statement)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MPI collective operations are one large source of interprocess communication. These operations have been optimized to minimize message counts for small messages, such as with Recursive-Doubling algorithms [12] and the Bruck algorithm [13]. Furthermore, collectives with large message sizes have been optimized to reduce the number of bytes transported between processes with algorithms such as the scatter-allgather and reduce-scatter [14], [15].…”

Section: Introductionmentioning

confidence: 99%

Modeling Data Movement Performance on Heterogeneous Architectures

Bienz¹,

Olson²,

Gropp³

et al. 2020

Preprint

View full text Add to dashboard Cite

The cost of data movement on parallel systems varies greatly with machine architecture, job partition, and even nearby jobs. Performance models that accurately capture the cost of data movement provide a tool for analysis, allowing for communication bottlenecks to be pinpointed. Modern heterogeneous architectures yield increased variance in data movement as there are a number of viable paths for inter-GPU communication.In this paper, we present performance models for the various paths of inter-node communication on modern heterogeneous architectures. We model the performance of utilizing all available CPU cores as well as the benefit of copying data to the CPUs when sending many messages. Finally, we present optimizations for a variety of MPI collectives based on the performance expectations provided by these models.

show abstract

Section: Introductionmentioning

confidence: 99%

Modeling Data Movement Performance on Heterogeneous Architectures

Bienz¹,

Olson²,

Gropp³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

Software Defined Multicasting for MPI Collective Operation Offloading with the NetFPGA

Arap

Brown

Himebaugh

et al. 2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Process arrival pattern aware algorithms for acceleration of scatter and gather operations

Proficz

2020

Cluster Comput

View full text Add to dashboard Cite

Imbalanced process arrival patterns (PAPs) are ubiquitous in many parallel and distributed systems, especially in HPC ones. The collective operations, e.g. in MPI, are designed for equal process arrival times, and are not optimized for deviations in their appearance. We propose eight new PAP-aware algorithms for the scatter and gather operations. They are binomial or linear tree adaptations introducing additional process ordering and (in some cases) additional activities in a special background thread. The solution was implemented using one of the most popular open source MPI compliant library (OpenMPI), and evaluated in a typical HPC environment using a specially developed benchmark as well as a real application: FFT. The experimental results show a significant advantage of the proposed approach over the default OpenMPI implementation, showing good scalability and high performance with the FFT acceleration for the communication run time: 16.7% and for the total application execution time: 3.3%.

show abstract

Adaptive Recursive Doubling Algorithm for Collective Communication

Cited by 7 publications

References 13 publications

Modeling Data Movement Performance on Heterogeneous Architectures

Modeling Data Movement Performance on Heterogeneous Architectures

Software Defined Multicasting for MPI Collective Operation Offloading with the NetFPGA

Process arrival pattern aware algorithms for acceleration of scatter and gather operations

Contact Info

Product

Resources

About