Hybrid MPI

Friedley, Andrew; Bronevetsky, Greg; Hoefler, Torsten; Lumsdaine, Andrew

doi:10.1145/2503210.2503294

Cited by 35 publications

(1 citation statement)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This can result in significantly faster data accesses in shared memory architectures [3,7,9,11,16,33]. On the other hand, data used by each MPI rank is generally private to the rank [14], such that locality issues have a much lower impact on a single cluster node in general.…”

Section: Introductionmentioning

confidence: 99%

Improving the memory access locality of hybrid MPI applications

Diener

White

Kalé

et al. 2017

Proceedings of the 24th European MPI Users' Group Meeting

View full text Add to dashboard Cite

Maintaining memory access locality is continuing to be a challenge for parallel applications and their runtime environments. By exploiting locality, application performance, resource usage, and performance portability can be improved. The main challenge is to detect and fix memory locality issues for applications that use shared-memory programming models for intra-node parallelization. In this paper, we investigate improving memory access locality of a hybrid MPI+OpenMP application in two different ways, by manually fixing locality issues in its source code and by employing the Adaptive MPI (AMPI) runtime environment. Results show that AMPI can result in similar locality improvements as manual source code changes, leading to substantial performance and scalability gains compared to the unoptimized version and to a pure MPI runtime. Compared to the hybrid MPI+OpenMP baseline, our optimizations improved performance by 1.8x on a single cluster node, and by 1.4x on 32 nodes, with a speedup of 2.4x compared to a pure MPI execution on 32 nodes. In addition to performance, we also evaluate the impact of memory locality on the load balance within a node. CCS CONCEPTS • Computer systems organization → Multicore architectures; Distributed architectures; • Software and its engineering → Main memory; Runtime environments;

show abstract

Section: Introductionmentioning

confidence: 99%

Improving the memory access locality of hybrid MPI applications

Diener

White

Kalé

et al. 2017

Proceedings of the 24th European MPI Users' Group Meeting

View full text Add to dashboard Cite

show abstract

Optimizing point‐to‐point communication between adaptive MPI endpoints in shared memory

White

Kalé

2018

Concurrency and Computation

View full text Add to dashboard Cite

Adaptive MPI is an implementation of the MPI standard that supports the virtualization of ranks as user-level threads, rather than OS processes. In this work, we optimize the communication performance of AMPI based on the locality of the endpoints communicating within a cluster of SMP nodes. We differentiate between point-to-point messages with both endpoints co-located on the same execution unit and point-to-point messages with both endpoints residing in the same process but not on the same execution unit. We demonstrate how the messaging semantics of Charm++ enable and hinder AMPI's implementation in different ways, and motivate extensions to Charm++ to address the limitations. Using the OSU microbenchmark suite, we show that our locality-aware design offers lower latency, higher bandwidth, and reduced memory footprint for applications.

show abstract

A methodology for assessing computation/communication overlap of MPI nonblocking collectives

Denis

Jaeger

Jeannot

et al. 2022

Concurrency and Computation

View full text Add to dashboard Cite

Summary By allowing computation/communication overlap, MPI nonblocking collectives (NBC) are supposed to improve application scalability and performance. However, it is known that to actually get overlap, the MPI library has to implement progression mechanisms in software or rely on the network hardware. These mechanisms may be present or not, adequate or perfectible, they may have an impact on communication performance or may interfere with computation by stealing CPU cycles. From a user point of view, assessing and understanding the behavior of an MPI library concerning computation/communication overlap is difficult. In this article, we propose a methodology to assess the computation/communication overlap of NBC. We propose new metrics to measure how much communication and computation do overlap, and to evaluate how they interfere with each other. We integrate these metrics into a complete methodology. We compare our methodology with state of the art metrics and benchmarks, and show that ours provides more meaningful informations. We perform experiments on a large panel of MPI implementations and network hardware and show when and why overlap is efficient, nonexistent or even degrades performance.

show abstract

Hybrid MPI

Cited by 35 publications

References 13 publications

Improving the memory access locality of hybrid MPI applications

Improving the memory access locality of hybrid MPI applications

Optimizing point‐to‐point communication between adaptive MPI endpoints in shared memory

A methodology for assessing computation/communication overlap of MPI nonblocking collectives

Contact Info

Product

Resources

About