Increasing the Efficiency of Sparse Matrix-Matrix Multiplication with a 2.5D Algorithm and One-Sided MPI

Abstract

Matrix-matrix multiplication is a basic operation in linear algebra and an essential building block for a wide range of algorithms in various scienti c elds. eory and implementation for the dense, square matrix case are well-developed. If matrices are sparse, with application-speci c sparsity pa erns, the optimal implementation remains an open question. Here, we explore the performance of communication reducing 2.5D algorithms and one-sided MPI communication in the context of linear scaling electronic structure theory. In particular, we extend the DBCSR sparse matrix library, which is the basic building block for linear scaling electronic structure theory and low scaling correlated methods in CP2K. e library is speci cally designed to e ciently perform block-sparse matrix-matrix multiplication of matrices with a relatively large occupation. Here, we compare the performance of the original implementation based on Cannon's algorithm and MPI point-to-point communication, with an implementation based on MPI one-sided communications (RMA), in both a 2D and a 2.5D approach. e 2.5D approach trades memory and auxiliary operations for reduced communication, which can lead to a speedup if communication is dominant. e 2.5D algorithm is somewhat easier to implement with one-sided communications. A detailed description of the implementation is provided, also for non ideal processor topologies, since this is important for actual applications. Given the importance of the precise sparsity pa ern, and even the actual matrix data, which decides the e ective ll-in upon multiplication, the tests are performed within the CP2K package with application benchmarks. Results show a substantial boost in performance for the RMA based 2.5D algorithm, up to 1.80x, which is observed to increase with the number of involved processes in the parallelization.•Computing methodologies →Massively parallel algorithms; •Applied computing →Chemistry; •So ware and its engineering →So ware libraries and repositories;

Sparse matrix multiplication and triangle listing in the Congested Clique model

“…Solomon and Demmel [28] give a 2.5-dimensional matrix multiplication algorithm, in the sense that the cube is split to (n/c) 1/2 × (n/c) 1/2 × c sub-cubes. A recent work by Lazzaro et al [23] provides a 2.5D algorithm that is also suitable for sparse matrices, which also employs random permutations of rows and columns.…”

Improved Unconstrained Energy Functional Method for Eigensolvers in Electronic Structure Calculations

“…e drop o in performance for large node counts is caused by the increased communication/computation ratios, particularly for the sparse linear algebra operations. In this respect, the recent progress in the use of one-sided MPI and a 2.5D algorithm to reduce communication in the DBCSR [45] are promising avenues to improve parallel scalability. Weak scaling performance for these types of electronic structure methods is more di cult to quantify.…”

ELSI — An open infrastructure for electronic structure solvers

“…The 2 nd order trace resetting density matrix purification method was ported to GPUs [87] by using matrix multiplications provided in the cuBLAS library [88]. Similarly, the GPU-accelerated sparse matrix linear algebra routines in the DBCSR library [89,90] can be employed to compute the density matrix on GPUs.…”