Accelerating CPU-Based Sparse General Matrix Multiplication With Binary Row Merging

Abstract

Sparse general matrix multiplication (SpGEMM) is a fundamental building block for many realworld applications. Since SpGEMM is a well-known memory-bounded application with vast and irregular memory accesses, considering the memory access efficiency is of critical importance for optimizing SpGEMM. Yet, the existing methods put less consideration into the memory subsystem and achieved suboptimal performance. In this paper, we thoroughly analyze the memory access patterns of SpGEMM and their influences on the memory subsystem. Based on the analysis, we propose a novel and more efficient accumulation method named BRMerge for the multi-core CPU architectures. The BRMerge accumulation method follows the row-wise dataflow. It first accesses the B matrix, generates the intermediate lists for one output row, and stores these intermediate lists in a consecutive memory space, which is implemented by a ping-pong buffer. It then immediately merges these intermediate lists generated in the previous phase two by two in a tree-like hierarchy between two ping-pong buffers. The architectural benefits of BRMerge are 1) streaming access patterns, 2) minimized TLB cache misses, and 3) reasonably high L1/L2 cache hit rates, which result in both low access latency and high bandwidth utilization when performing SpGEMM. Based on the BRMerge accumulation method, we propose two SpGEMM libraries named BRMerge-Upper and BRMerge-Precise, which use different allocation methods. Performance evaluations with 26 commonly used benchmarks on two CPU servers show that the proposed SpGEMM libraries significantly outperform the state-of-the-art SpGEMM libraries.

Predicting the Output Structure of Sparse Matrix Multiplication with Sampled Compression Ratio

“…For example, both input matrices may be stored in the CSR format, which is hard to select the columns from the matrix. Furthermore, even if the rows and columns can be selected with a low cost, the following inner-product dataflow may still be a performance bottleneck [16], [17]. To tackle these two performance issues, we adopt the row-wise dataflow for the sampling method and the computation of the samples.…”

“…In this section, we compare the accuracy of the predicted NNZ(C) of the proposed method and the reference design. We also show the prediction overheads of the proposed method 28.34 compared with a state-of-the-art SpGEMM library BRMerge-Precise [16].…”

“…As for the computing overheads, the existing samplingbased method uses the inner-product dataflow [9] to compute the sampled matrices, which is less efficient than the row-wise dataflow [16], [17] on both the computing and sampling of the two input matrices. Therefore, in this work, we implement the proposed method using the row-wise dataflow.…”

“…The relative prediction errors of the proposed method and the reference design are 1.56% and 8.12% (smaller is better), respectively, on average, and 25% and 156%, respectively, in the worst case. We also conduct experiments to show that the parallel implementation of the proposed method only takes on average 0.78% execution time of an entire state-of-the-art SpGEMM library (BRMerge-Precise) [16].…”

“…• We conduct comprehensive evaluations to show that the proposed method is much more accurate than the reference design of the existing method. We also conduct experiments to show that the parallel implementation of the proposed method only takes on average 0.78% execution time of an entire state-of-the-art SpGEMM library (BRMerge-Precise) [16].…”