Improving the memory access locality of hybrid MPI applications

Abstract

Maintaining memory access locality is continuing to be a challenge for parallel applications and their runtime environments. By exploiting locality, application performance, resource usage, and performance portability can be improved. The main challenge is to detect and fix memory locality issues for applications that use shared-memory programming models for intra-node parallelization. In this paper, we investigate improving memory access locality of a hybrid MPI+OpenMP application in two different ways, by manually fixing locality issues in its source code and by employing the Adaptive MPI (AMPI) runtime environment. Results show that AMPI can result in similar locality improvements as manual source code changes, leading to substantial performance and scalability gains compared to the unoptimized version and to a pure MPI runtime. Compared to the hybrid MPI+OpenMP baseline, our optimizations improved performance by 1.8x on a single cluster node, and by 1.4x on 32 nodes, with a speedup of 2.4x compared to a pure MPI execution on 32 nodes. In addition to performance, we also evaluate the impact of memory locality on the load balance within a node. CCS CONCEPTS • Computer systems organization → Multicore architectures; Distributed architectures; • Software and its engineering → Main memory; Runtime environments;

An evaluation of MPI and OpenMP paradigms in finite‐difference explicit methods for PDEs on shared‐memory multi‐ and manycore systems

“…In both solutions, scaling was much better than executing the pure MPI‐based implementation. With this experiment, Diener et al observed the influence of memory locality on load balance, A better locality of reference leads to lower stall time due to memory access, resulting in reasonable use of the resources. The same authors reported that AMPI leads to similar locality gains, achieving better performance and scalability than the simplistic hybrid MPI/OpenMP and pure MPI‐based implementations.…”

“…For a similar purpose as other investigations, this study analyzed hybrid configurations to evaluate the impact of message passing operations for a large number of processes. Similar to investigations conducted in previous publications, this study evaluated the performance of a hybrid MPI/OpenMP approach concerning load‐balancing and synchronization techniques.…”

“…Adaptive message passing interface (AMPI) supports dynamic load balancing, processor virtualization, and fault tolerance for MPI applications. Diener et al investigated how to improve the memory access locality of a hybrid MPI/OpenMP multiphysics simulation application in two different strategies: manually fixing locality (NUMA) issues and by using the AMPI runtime environment. Both strategies showed different trade‐offs.…”

“…Similar to the publications described in this section, the present study evaluated parallel models of methods when running on multicore and manycore architectures. Along the same lines of the investigations conducted in previous publications, we implemented hybrid approaches to reduce the NUMA effect. For a similar purpose as other investigations, this study analyzed hybrid configurations to evaluate the impact of message passing operations for a large number of processes.…”

Otimização do Método HOPMOC 1D com auxílio das ferramentas Intel Parallel Studio

“…Trabalhos com objetivos parecidos já foram desenvolvidos para outras aplicações, como [Diener et al 2017] em um estudo para aperfeiçoar o acessoà memória em aplicações híbridas MPI/OpenMP e compará-lo com as versões padrão e puramente MPI, e [Bassi et al 2016] que apresenta um modelo de paralelização híbrido MPI/OpenMP para o método Galerkin Descontínuo. Os dois trabalhos desenvolvem a otimização MPI/OpenMP também em arquitetura com memória compartilhada.…”