MPI and Compiler Technology: A Love-Hate Relationship

Danalis, Anthony

doi:10.1007/978-3-642-33518-1_4

“…The MPI library standard 1.0 was ratified on May 5th 1994, and yet after 20 years and multiple compiled MPI efforts [14,17,21,29], synchronization optimizations are performed manually in codes. The UPC language standard 1.0 has been ratified on May 13th 1999, yet after 15 years of compiler development [1,3,18] we are in a similar position.…”

Section: Introductionmentioning

confidence: 99%

Barrier elision for production parallel programs

Chabbi

¹

,

Lavrijsen

²

,

Jong

³

et al. 2015

Proceedings of the 20th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming

View full text Add to dashboard Cite

Large scientific code bases are often composed of several layers of runtime libraries, implemented in multiple programming languages. In such situation, programmers often choose conservative synchronization patterns leading to suboptimal performance. In this paper, we present context-sensitive dynamic optimizations that elide barriers redundant during the program execution. In our technique, we perform data race detection alongside the program to identify redundant barriers in their calling contexts; after an initial learning, we start eliding all future instances of barriers occurring in the same calling context. We present an automatic on-the-fly optimization and a multi-pass guided optimization. We apply our techniques to NWChem-a 6 million line computational chemistry code written in C/C++/Fortran that uses several runtime libraries such as Global Arrays, ComEx, DMAPP, and MPI. Our technique elides a surprisingly high fraction of barriers (as many as 63%) in production runs. This redundancy elimination translates to application speedups as high as 14% on 2048 cores. Our techniques also provided valuable insight about the application behavior, later used by NWChem developers. Overall, we demonstrate the value of holistic context-sensitive analyses that consider the domain science in conjunction with the associated runtime software stack.

show abstract

Combining XOR and Partner Checkpointing for Resilient Multilevel Checkpoint/Restart

Gholami

¹

,

Schintke

²

2021

2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS)

1

0

View full text Add to dashboard Cite

Barrier elision for production parallel programs

Chabbi

¹

,

Lavrijsen

²

,

Jong

³

et al. 2015

View full text Add to dashboard Cite

Large scientific code bases are often composed of several layers of runtime libraries, implemented in multiple programming languages. In such situation, programmers often choose conservative synchronization patterns leading to suboptimal performance. In this paper, we present context-sensitive dynamic optimizations that elide barriers redundant during the program execution. In our technique, we perform data race detection alongside the program to identify redundant barriers in their calling contexts; after an initial learning, we start eliding all future instances of barriers occurring in the same calling context. We present an automatic on-the-fly optimization and a multi-pass guided optimization. We apply our techniques to NWChem-a 6 million line computational chemistry code written in C/C++/Fortran that uses several runtime libraries such as Global Arrays, ComEx, DMAPP, and MPI. Our technique elides a surprisingly high fraction of barriers (as many as 63%) in production runs. This redundancy elimination translates to application speedups as high as 14% on 2048 cores. Our techniques also provided valuable insight about the application behavior, later used by NWChem developers. Overall, we demonstrate the value of holistic context-sensitive analyses that consider the domain science in conjunction with the associated runtime software stack.

show abstract

MPI and Compiler Technology: A Love-Hate Relationship

Cited by 3 publications

References 6 publications

Barrier elision for production parallel programs

Barrier elision for production parallel programs

Combining XOR and Partner Checkpointing for Resilient Multilevel Checkpoint/Restart

Barrier elision for production parallel programs

Contact Info

Product

Resources

About