Enabling communication concurrency through flexible MPI endpoints

Dinan, James; Grant, Ryan E.; Balaji, Pavan; Goodell, David; Miller, Douglas; Snir, Marc; Thakur, Rajeev

doi:10.1177/1094342014548772

Cited by 46 publications

(19 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, because MPC is built on user-level threading system, not only does our approach track process-based MPI, but it can accommodate any type of thread-based MPI, including user-level threads in MPC. Thus, this demonstration in MPC gives us confidence that the methodology will translate well to future evolutions of MPI, including those supporting the concepts of endpoints [14] and sessions [21], which involve intra-process parallelism.…”

Section: Dmtcp Supportmentioning

confidence: 93%

Checkpoint/restart approaches for a thread-based MPI runtime

Adam¹,

Kermarquer

Besnard³

et al. 2019

Parallel Computing

View full text Add to dashboard Cite

Fault-tolerance has always been an important topic when it comes to running massively parallel programs at scale. Statistically, hardware and software failures are expected to occur more often on systems gathering millions of computing units. Moreover, the larger jobs are, the more computing hours would be wasted by a crash. In this paper, we describe the work done in our MPI runtime to enable both transparent and application-level checkpointing mechanisms. Unlike the MPI 4.0 User-Level Failure Mitigation (ULFM) interface, our work targets solely Checkpoint/Restart and ignores other features such as resiliency. We show how existing checkpointing methods can be practically applied to a thread-based MPI implementation given sufficient runtime collaboration. The two main contributions are the preservation of high-speed network performance during transparent C/R and the over-subscription of checkpoint data replication thanks to a dedicated user-level scheduler support. These techniques are measured on MPI benchmarks such as IMB, Lulesh and Heatdis, and associated overhead and trade-offs are discussed.

show abstract

Section: Dmtcp Supportmentioning

confidence: 93%

Checkpoint/restart approaches for a thread-based MPI runtime

Adam¹,

Kermarquer

Besnard³

et al. 2019

Parallel Computing

View full text Add to dashboard Cite

show abstract

“…The MPI endpoints extension [6,18] is intended to address similar challenges in the context of MPI by providing additional ranks that can be assigned to threads. A significant difference between endpoints and contexts is that endpoints are individually addressable in MPI communication operations.…”

Section: Related Workmentioning

confidence: 99%

Contexts

Dinan¹,

Flajšlik²

2014

Proceedings of the 8th International Conference on Partitioned Global Address Space Programming Models

Self Cite

View full text Add to dashboard Cite

This paper introduces a proposed extension to the OpenSHMEM parallel programming model, called communication contexts. Contexts introduce a new construct that allows a programmer to generate independent streams of communication operations. In hybrid executions where multiple threads execute within an Open-SHMEM process, contexts eliminate interference between threads, and enable the OpenSHMEM library to map operations generated by threads to private communication resource sets. By providing thread isolation, contexts eliminate synchronization overheads and enable each thread to drive a similar set of resources and achieve performance comparable to an OpenSHMEM process. In conventional, single-threaded execution, contexts provide greater control over ordering of operations and can improve communication and computation overlap. A detailed description of the contexts interface and its implementation for the Portals 4 network programming interface is described. The implementation is evaluated using Mandelbrot set and integer sorting (IS) benchmarks. Contexts provide a 25% performance improvement for Mandelbrot by eliminating thread interference and enabling pipelining, and a 35% improvement was achieved for IS by enabling more effective communication/computation overlap.

show abstract

“…MPI endpoints. Dinan et al [14] enable multiple communication endpoints by creating additional MPI ranks that serve as the "MPI endpoints." The threads within the MPI ranks then map to the MPI endpoints, achieving the same configuration as MPI-everywhere endpoints since each MPI endpoint has its CTX.…”

Section: Related Workmentioning

confidence: 99%

Scalable Communication Endpoints for MPI+Threads Applications

Zambre

Chandramowlishwaran

Balaji

2018

2018 IEEE 24th International Conference on Parallel and Distributed Systems (ICPADS)

Self Cite

View full text Add to dashboard Cite

Hybrid MPI+threads programming is gaining prominence as an alternative to the traditional "MPI everywhere" model to better handle the disproportionate increase in the number of cores compared with other on-node resources. Current implementations of these two models represent the two extreme cases of communication resource sharing in modern MPI implementations. In the MPI-everywhere model, each MPI process has a dedicated set of communication resources (also known as endpoints), which is ideal for performance but is resource wasteful. With MPI+threads, current MPI implementations share a single communication endpoint for all threads, which is ideal for resource usage but is hurtful for performance.In this paper, we explore the tradeoff space between performance and communication resource usage in MPI+threads environments. We first demonstrate the two extreme cases-one where all threads share a single communication endpoint and another where each thread gets its own dedicated communication endpoint (similar to the MPI-everywhere model) and showcase the inefficiencies in both these cases. Next, we perform a thorough analysis of the different levels of resource sharing in the context of Mellanox InfiniBand. Using the lessons learned from this analysis, we design an improved resource-sharing model to produce scalable communication endpoints that can achieve the same performance as with dedicated communication resources per thread but using just a third of the resources.

show abstract

Enabling communication concurrency through flexible MPI endpoints

Cited by 46 publications

References 29 publications

Checkpoint/restart approaches for a thread-based MPI runtime

Checkpoint/restart approaches for a thread-based MPI runtime

Contexts

Scalable Communication Endpoints for MPI+Threads Applications

Contact Info

Product

Resources

About