Architecture of the Component Collective Messaging Interface

Kumar, Sameer; Faraj, Ahmad; Mamidala, Amith R.; Smith, Brian E.; Dózsa, Gábor; Cernohous, Bob; Gunnels, John A.; Miller, Douglas; Ratterman, Joe; Heidelberger, Philip

doi:10.1177/1094342009359011

Cited by 8 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The performance for large messages at PPN=4 and 16 saturates as the broadcast data spills out of the L2 cache and the performance is driven by DDR throughput. To improve broadcast performance, by up to a factor of nearly 10, we also implemented a 10-color rectangle broadcast, where the root sends data to all the remaining nodes in the 5D torus via 10 edge disjoint spanning trees [15]. The peak throughput of this algorithm is 18 GB/s.…”

Section: Performance Anaylysismentioning

confidence: 99%

PAMI: A Parallel Active Message Interface for the Blue Gene/Q Supercomputer

Kumar

Mamidala

Faraj

et al. 2012

2012 IEEE 26th International Parallel and Distributed Processing Symposium

Self Cite

View full text Add to dashboard Cite

The Blue Gene/Q machine is the next generation in the line of IBM massively parallel supercomputers, designed to scale to 262144 nodes and sixteen million threads. With each BG/Q node having 68 hardware threads, hybrid programming paradigms, which use message passing among nodes and multi-threading within nodes, are ideal and will enable applications to achieve high throughput on BG/Q. With such unprecedented massive parallelism and scale, this paper is a groundbreaking effort to explore the design challenges for designing a communication library that can match and exploit such massive parallelism In particular, we present the Parallel Active Messaging Interface (PAMI) library as our BG/Q library solution to the many challenges that come with a machine at such scale. PAMI provides (1) novel techniques to partition the application communication overhead into many contexts that can be accelerated by communication threads; (2) client and context objects to support multiple and different programming paradigms; (3) lockless algorithms to speed up MPI message rate; and (4) novel techniques leveraging the new BG/Q architectural features such as the scalable atomic primitives implemented in the L2 cache, the highly parallel hardware messaging unit that supports both point-to-point and collective operations, and the collective hardware acceleration for operations such as broadcast, reduce, and allreduce. We experimented with PAMI on 2048 BG/Q nodes and the results show high messaging rates as well as low latencies and high throughputs for collective communication operations.

show abstract

Section: Performance Anaylysismentioning

confidence: 99%

PAMI: A Parallel Active Message Interface for the Blue Gene/Q Supercomputer

Kumar

Mamidala

Faraj

et al. 2012

2012 IEEE 26th International Parallel and Distributed Processing Symposium

Self Cite

View full text Add to dashboard Cite

show abstract

“…Native Algorithm. Kumar et al [2010] describes the different native all-gather algorithms used on the Blue Gene/P.…”

Section: All-gather Algorithmsmentioning

confidence: 99%

Collective Algorithms for Multiported Torus Networks

Sack

Gropp

2015

ACM Trans. Parallel Comput.

View full text Add to dashboard Cite

Modern supercomputers with torus networks allow each node to simultaneously pass messages on all of its links. However, most collective algorithms are designed to only use one link at a time. In this work, we present novel multiported algorithms for the scatter, gather, all-gather, and reduce-scatter operations. Our algorithms can be combined to create multiported reduce, all-reduce, and broadcast algorithms. Several of these algorithms involve a new technique where we relax the MPI message-ordering constraints to achieve high performance and restore the correct ordering using an additional stage of redundant communication.According to our models, on an n-dimensional torus, our algorithms should allow for nearly a 2n-fold improvement in communication performance compared to known, single-ported torus algorithms. In practice, we have achieved nearly 6x better performance on a 32k-node 3-dimensional torus.

show abstract

“…PAMI collective library builds upon the portable Component Collective Messaging Interface (CCMI [4]) from Blue Gene/P, with optimized implementations on Blue Gene/Q, Power7 IH [6] and Intel x86 clusters.…”

Section: Pami Collectivesmentioning

confidence: 99%

Optimization of MPI_Allreduce on the blue Gene/Q supercomputer

Kumar

Faraj

2013

Proceedings of the 20th European MPI Users' Group Meeting

Self Cite

View full text Add to dashboard Cite

The IBM Blue Gene/Q supercomputer has a 5D torus network where each node is connected to ten bi-directional links. In this paper we present techniques to optimize the MPI Allreduce collective operation by building ten different edge disjoint spanning trees on the ten torus links. We accelerate summing of network packets with local buffers by the use of Quad Processing SIMD unit in the BG/Q cores and executing the sums on multiple communication threads created by the PAMI libraries. The net gain we achieve is a peak throughput of 6.3 GB/sec for double precision floating point sum allreduce, that is a speedup of 3.75x over the collective network based algorithm in the product MPI stack on BG/Q.

show abstract

Architecture of the Component Collective Messaging Interface

Cited by 8 publications

References 14 publications

PAMI: A Parallel Active Message Interface for the Blue Gene/Q Supercomputer

PAMI: A Parallel Active Message Interface for the Blue Gene/Q Supercomputer

Collective Algorithms for Multiported Torus Networks

Optimization of MPI_Allreduce on the blue Gene/Q supercomputer

Contact Info

Product

Resources

About