Tuning collective communication for Partitioned Global Address Space programming models

Nishtala, Rajesh; Zheng, Yili; Hargrove, Paul; Yelick, Katherine

doi:10.1016/j.parco.2011.05.006

Cited by 33 publications

(21 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MPI [22,17] and parallel programming languages such as UPC [19] provide optimized implementations of all-to-all collective operations. Most if not all of the existing implementations use multiple algorithms selected by message size.…”

Section: All-to-all Performancementioning

confidence: 99%

Congestion avoidance on manycore high performance computing systems

Luo

Panda

Ibrahim

et al. 2012

Proceedings of the 26th ACM International Conference on Supercomputing

View full text Add to dashboard Cite

Efficient communication is a requirement for application scalability on High Performance Computing systems. In this paper we argue for incorporating proactive congestion avoidance mechanisms into the design of communication layers on manycore systems. This is in contrast with the status quo which employs a reactive approach, e.g. congestion control mechanisms are activated only when resources have been exhausted. We present a core stateless optimization approach based on open loop end-point throttling, implemented for two UPC runtimes (Cray and Berkeley UPC) and validated on InfiniBand and the Cray Gemini networks. Microbenchmark results indicate that throttling the number of messages in flight per core can provide up to 4X performance improvements, while throttling the number of active cores per node can provide additional 40% and 6X performance improvement for UPC and MPI respectively. We evaluate inline (each task makes independent decisions) and proxy (server) congestion avoidance designs. Our runtime provides both performance and performance portability. We improve allto-all collective performance by up to 4X and provide better performance than vendor provided MPI and UPC implementations. We also demonstrate performance improvements of up to 60% in application settings. Overall, our results indicate that modern systems accommodate only a surprisingly small number of messages in flight per node. As Exascale projections indicate that future systems are likely to contain hundreds to thousands of cores per node, we believe that their networks will be underprovisioned. In this situation, proactive congestion avoidance might become mandatory for performance improvement and portability.

show abstract

Section: All-to-all Performancementioning

confidence: 99%

Congestion avoidance on manycore high performance computing systems

Luo

Panda

Ibrahim

et al. 2012

Proceedings of the 26th ACM International Conference on Supercomputing

View full text Add to dashboard Cite

show abstract

“…Berkeley UPC provides optimized multi-threaded collective communication for multi-core systems [12] implemented with the GASNet communication library [4]. The GASNet multi-threaded collectives utilize automated-tuning to select the appropriate algorithm and adapt the system parameters to achieve best performance [10]. UPC applications can take advantage of these software engineering efforts in Berkeley UPC by simply calling UPC collective functions with threads.…”

Section: Managing Data Affinity For Numa Systemsmentioning

confidence: 99%

Optimizing UPC Programs for Multi-Core Systems

Zheng

2010

Scientific Programming

Self Cite

View full text Add to dashboard Cite

Abstract. The Partitioned Global Address Space (PGAS) model of Unified Parallel C (UPC) can help users express and manage application data locality on non-uniform memory access (NUMA) multi-core shared-memory systems to get good performance. First, we describe several UPC program optimization techniques that are important to achieving good performance on NUMA multi-core computers with examples and quantitative performance results. Second, we use two numerical computing kernels, parallel matrix-matrix multiplication and parallel 3-D FFT, to demonstrate the end-to-end development and optimization for UPC applications. Our results show that the optimized UPC programs achieve very good and scalable performance on current multi-core systems and can even outperform vendor-optimized libraries in some cases.

show abstract

“…Support for asynchronous collectives is still under proposal. GASNet implements a range of non-blocking collective operations such as broadcast, reduce, scatter, gather, exchange and allgather with performance comparable to or better than MPI [16]. Aspen supports asynchronous collective communication [1].…”

Section: Related Workmentioning

confidence: 99%

Hiding latency in Coarray Fortran 2.0

Scherer

Adhianto

Jin

et al. 2010

Proceedings of the Fourth Conference on Partitioned Global Address Space Programming Model

View full text Add to dashboard Cite

In Numrich and Reid's 1998 proposal [17], Coarray Fortran is a simple set of extensions to Fortran 95, principal among which is support for shared data known as coarrays. Responding to shortcomings in the Fortran Standards Committee's addition of coarrays to the Fortran 2008 standards, we at Rice envisioned an extensive update which has come to be known as Coarray Fortran 2.0 [15]. In this paper, we chronicle the evolution of Coarray Fortran 2.0 as it gains support for asynchronous point-to-point and collective operations. We outline how these operations are implemented and describe code fragments from several benchmark programs to show we use these operations to hide latency by overlapping communication and computation.

show abstract

Tuning collective communication for Partitioned Global Address Space programming models

Cited by 33 publications

References 9 publications

Congestion avoidance on manycore high performance computing systems

Congestion avoidance on manycore high performance computing systems

Optimizing UPC Programs for Multi-Core Systems

Hiding latency in Coarray Fortran 2.0

Contact Info

Product

Resources

About