A framework for general sparse matrix–matrix multiplication on GPUs and heterogeneous processors

Abstract

h i g h l i g h t s• We design a framework for SpGEMM on modern manycore processors using the CSR format.• We present a hybrid method for pre-allocating the resulting sparse matrix.• We propose an efficient parallel insert method for long rows of the resulting matrix.• We develop a heuristic-based load balancing strategy. • Our approach significantly outperforms other known CPU and GPU SpGEMM methods. a b s t r a c tGeneral sparse matrix-matrix multiplication (SpGEMM) is a fundamental building block for numerous applications such as algebraic multigrid method (AMG), breadth first search and shortest path problem. Compared to other sparse BLAS routines, an efficient parallel SpGEMM implementation has to handle extra irregularity from three aspects: (1) the number of nonzero entries in the resulting sparse matrix is unknown in advance, (2) very expensive parallel insert operations at random positions in the resulting sparse matrix dominate the execution time, and (3) load balancing must account for sparse data in both input matrices.In this work we propose a framework for SpGEMM on GPUs and emerging CPU-GPU heterogeneous processors. This framework particularly focuses on the above three problems. Memory pre-allocation for the resulting matrix is organized by a hybrid method that saves a large amount of global memory space and efficiently utilizes the very limited on-chip scratchpad memory. Parallel insert operations of the nonzero entries are implemented through the GPU merge path algorithm that is experimentally found to be the fastest GPU merge approach. Load balancing builds on the number of necessary arithmetic operations on the nonzero entries and is guaranteed in all stages.Compared with the state-of-the-art CPU and GPU SpGEMM methods, our approach delivers excellent absolute performance and relative speedups on various benchmarks multiplying matrices with diverse sparsity structures. Furthermore, on heterogeneous processors, our SpGEMM approach achieves higher throughput by using re-allocatable shared virtual memory.

Benchmarking overlapping communication and computations with multiple streams for modern GPUs

“…Notable applications using such operations nowadays include, for example, machine learning [14], weather pattern analsis [15], shortest path problem etc. [16]. 2) Various compute intensities correspond to various operations such as: higher for matrix multiplication, lower for addition of vectors etc.…”

Register-based implementation of the sparse general matrix-matrix multiplication on GPUs

“…We use Nvidia K40m (Kepler) and Titan X (Pascal) GPUs for comparing the performance of our algorithm and several existing methods (CUSP [1], cuSPARSE, bhSPARSE [5] and RMerge [3]) that compute C = A 2 in double precision. The CUDA versions are 7.0 and 8.0 on K40m and Titan X, respectively.…”

A GPU-Adapted Structure for Unstructured Grids

“…The biggest challenges are (i) that the structure of the resulting matrix depends on the input matrices, (ii) that the organization of the entries in the resulting matrix requires communication between threads, and (iii) that the number of operations carried out by individual threads may vary strongly. To provide an efficient implementation we take advantage of the algorithmic description of bhSparse [LV15]. We tackle the aforementioned issues in a four stage approach: In the first stage, we compute an upper bound for the number for nonzeros in each column of the result matrix, which allows for allocating sufficient storage.…”

“…Note that M v implies summation on the compressed direction. For reference, we report the timings for cuSparse [NVI15] and Bhsparse[LV15].…”