Impact of Kernel-Assisted MPI Communication over Scientific Applications: CPMD and FFTW

Abstract

Abstract. Collective communication is one of the most powerful message passing concepts, enabling parallel applications to express complex communication patterns while allowing the underlying MPI to provide efficient implementations to minimize the cost of the data movements. However, with the increase in the heterogeneity inside the nodes, more specifically the memory hierarchies, harnessing the maximum compute capabilities becomes increasingly difficult. This paper investigates the impact of kernel-assisted MPI communication, over two scientific applications: 1) Car-Parrinello molecular dynamics(CPMD), a chemical molecular dynamics application, and 2) FFTW, a Discrete Fourier Transform (DFT). By focusing on the usage of Message Passing Interface (MPI), we found the communication characteristics and patterns of each application. Our experiments indicate that the quality of the collective communication implementation on a specific machine plays a critical role on the overall application performance.

KNEM: A generic and scalable kernel-assisted intra-node MPI communication framework

“…Our NAS results showed that KNEM may significantly help real application performance by improving point-to-point operations while ASP experiments show that the native KNEM collective operations bring further significant efficiency. Other applications such as CPMD or FFTW have been recently studied and showed similar improvements thanks to KNEM [30].…”

“…KNEM brings significant throughput improvement to MPI communication on modern computing platforms because its single copy model reduces the CPU load, cache pollution, and memory bandwidth waste. It is already supported by most existing MPI implementations [3,5] and the use of its native interface for collective operations in Open MPI shows promising results in real world applications [30,37]. KNEM only benefits to intra-node communication, but it obviously adds a significant value to the general distributed case where both intra-node and inter-node communication are involved.…”

An In-Depth Analysis of the Slingshot Interconnect

“…Fast Fourier Transform on a 3D domain [39]. It employs broadcasts, scatters, and point-to-point communications [40].…”