Efficient Implementation of MPI-2 Passive One-Sided Communication on InfiniBand Clusters

Jiang, Weihang; Liu, Jiuxing; Jin, Hyun-Wook; Panda, Dhabaleswar K.; Buntinas, Darius; Thakur, Rajeev; Gropp, William

doi:10.1007/978-3-540-30218-6_16

Cited by 20 publications

(16 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We insert a win set info call with symmetric info at the Global Aarrays GA_SYNC call 5 , since its semantics guarantee the completion of all outstanding operations on all processes. We note that Casper(GP) requires three kinds of predefined thresholds: LOW|HIGH_CR, PREDICT_INT, and GSYNC_INT.…”

Section: Methodsmentioning

confidence: 99%

“…It provides a good mix of efficiency and accuracy to investigate the structural and electronic properties of atoms and molecules. It contains only a single 5. GA_SYNC internally calls MPI flush all on all processes followed by a barrier.…”

Section: Single-phase Dftmentioning

confidence: 99%

“…The reason is that even on RDMA-supported networks such as InfiniBand and the Cray Aries interconnect, most The traditional approaches to ensure asynchronous progress for MPI communication have relied on two models. One is to utilize the background threads dedicated to each MPI process in order to handle incoming messages from other processes [5]. This model is widely provided in mainstream MPI implementations [6], [7], [8], [9].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Dynamic Adaptable Asynchronous Progress Model for MPI RMA Multiphase Applications

Peña

Hammond

et al. 2018

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

Abstract-Casper is a process-based asynchronous progress model for MPI one-sided communication on multi-and many-core architectures. The one-sided communication is not yet truly one-sided in most MPI implementations: the target process still relies on software progress to complete incoming operations. Casper allows the user to specify an arbitrary number of cores dedicated to background ghost processes and transparently redirects the user RMA operations to ghost processes by utilizing the PMPI redirection and MPI-3 shared-memory technologies. Although Casper is effective and efficient for applications suffering from lack of asynchronous progress, permanently redirecting operations to a small number of ghost processes might not support complex multiphase applications effectively, which often involve dynamically changing communication density and computing workloads. In this paper, we present an adaptive mechanism in Casper to address the limitation of static asynchronous progress in multiphase applications. We exploit two adaptive strategies, a precise user-guided strategy and a fully transparent and automatic strategy based on self-profiling and prediction, to dynamically reconfigure the asynchronous progress in Casper according to real-time performance characteristics during multiphase execution. We evaluate the adaptive approaches in both microbenchmarks and a real quantum chemistry application suite, NWChem, on the Cray XC30 supercomputer and an Intel Omni-Path cluster.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Single-phase Dftmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Dynamic Adaptable Asynchronous Progress Model for MPI RMA Multiphase Applications

Peña

Hammond

et al. 2018

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

show abstract

“…The authors in [51] have studied different optimization for reducing the synchronization overhead involved in implementing MPI one-sided communication. Designs for MPI RMA in InfiniBand clusters has been described in [52] and [53].In [54] and [55], the author describes a design for efficient passive synchronization using hardware support from InfiniBand atomic operations. In [56], authors discuss some performance guidelines for one-sided communication in MPI.…”

Section: Related Workmentioning

confidence: 99%

Runtime Support for Irregular Computation in MPI-Based Applications

Zhao

Balaji

Gropp

2015

2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

View full text Add to dashboard Cite

In recent years there are increasing number of applications that have been using irregular computation models in various domains, such as computational chemistry, bioinformatics, nuclear reactor simulation and social network analysis. Due to the irregular and datadependent communication patterns and sparse data structures involved in those applications, the traditional parallel programming model and runtime need to be carefully designed and implemented in order to accommodate the performance and scalability requirements of those irregular applications on large-scale systems. This thesis consists of two subtopics. The first subtopic focuses on improving MPI runtime to support the irregular applications from perspective of scalability and performance. The first three parts in this subtopic focus on MPI one-sided communication. In the first part, we present a thorough survey of current MPI one-sided implementations and illustrate scalability limitations in those implementations. In the second part, we propose a new design and implementation of MPI one-sided communication, called ScalaRMA, to effectively address those scalability limitations. The third part in this subtopic focuses on various issuing strategies in MPI one-sided communication. We propose an adaptive issuing strategy which can adaptively choose between delayed issuing strategy and eager issuing strategy in MPI ii runtime to achieve high performance based on current communication volume in MPI-based application. The last part in this subtopic is to tackle the scalability limitations in the virtual connection (VC) objects in MPI implementation. We propose a scalable design to reduce the memory consumption of VC objects in MPI runtime.The second subtopic of this thesis focuses on improving MPI programming model to better support the irregular applications. Traditional two-sided data movement model in MPI standard designed for scientific computation provides a paradigm for user to specify how to move the data between processes, however, it does not provide interface to flexibly manage the computation, which means user needs to explicitly manage where the computation should be performed. This model is not well suited for irregular applications which involve irregular and data-dependent communication pattern. In this work, we combine Active Messages (AM), an alternative programming paradigm which is more suitable for irregular computations, with traditional MPI data movement model, and propose a generalized MPI-interoperable Active Messages framework (MPI-AM). The framework allows MPI-based applications to incrementally use AMs only when necessary, avoiding rewriting the entire MPI-based application. Such framework integrates data movement and computation together in the programming model and MPI can coordinate the computation and communication in a much more flexible manner. In this subtopic, we propose several strategies including message streaming, buffer management and asynchronous processing, in order to efficiently handle AMs inside MPI. We al...

show abstract

“…These data-transfer functions must be used together with one of three synchronization mechanismsfence, post-start-complete-wait, and lock-unlock-as shown in Figure 1. Many MPI implementations, including all vendor MPIs, support one-sided communication, with varying levels of optimization [1,2,4,6,7,9,10,12,14,15]. Nonetheless, Gabriel et al [3] found that, because of the synchronization overhead in one-sided communication, regular point-to-point communication performs better than one-sided communication in five MPI implementations: NEC, Hitachi, IBM, Sun, and LAM.…”

Section: Introductionmentioning

confidence: 99%

An Evaluation of Implementation Options for MPI One-Sided Communication

Gropp

Thakur

2005

Recent Advances in Parallel Virtual Machine and Message Passing Interface

Self Cite

View full text Add to dashboard Cite

Abstract. MPI defines one-sided communication operations-put, get, and accumulate-together with three different synchronization mechanisms that define the semantics associated with the initiation and completion of these operations. In this paper, we analyze the requirements imposed by the MPI Standard on any implementation of one-sided communication. We discuss options for implementing the synchronization mechanisms and analyze the cost associated with each. An MPI implementer can use this information to select the implementation method that is best suited (has the lowest cost) for a particular machine environment. We also report on experiments we ran on a Linux cluster and a Sun SMP to determine the gap between the performance that could be achievable and what is actually achieved with MPI.

show abstract

Efficient Implementation of MPI-2 Passive One-Sided Communication on InfiniBand Clusters

Cited by 20 publications

References 9 publications

Dynamic Adaptable Asynchronous Progress Model for MPI RMA Multiphase Applications

Dynamic Adaptable Asynchronous Progress Model for MPI RMA Multiphase Applications

Runtime Support for Irregular Computation in MPI-Based Applications

An Evaluation of Implementation Options for MPI One-Sided Communication

Contact Info

Product

Resources

About