Run-time optimization of sparse matrix-vector multiplication on SIMD machines

Abstract

Abstract. Sparse matrix-vector multiplication forms the heart of iterative linear solvers used widely in scientific computations (e.g., finite element methods). In such solvers, the matrix-vector product is computed repeatedly, often thousands of times, with updated values of the vector until convergence is achieved. In an SIMD architecture, each processor has to fetch the updated off-processor vector elements while computing its share of the product. In this paper, we report on run-time optimization of array distribution and offprocessor data fetching to reduce both the communication and computation time. The optimization is applied to a sparse matrix stored in a compressed sparse row-wise format. Actual runs on test matrices produced up to a 35 percent relative improvement over a block distribution with a naive multiplication algorithm while simulations over a wider range of processors indicate that up to a 60 percent improvement may be possible in some cases.

Performance evaluation of data distributions with load-balancing for sparse arrays

“…All programs are written in C + MPI (Message Passing Interface) [21] codes. The sparse ratio is set to 0.1 for all test three-dimensional sparse arrays used as test samples.…”

“…Ziantz et al [21] proposed a run-time technique that was applied to sparse arrays for array distributions and off-processor data fetching to reduce the communication and computation time. They used the block data distribution scheme with a bin-packing algorithm to distribute a global sparse array to processors.…”

“…However, it is a challenging problem to provide an efficient data distribution for irregular problems [18] on distributed memory multicomputers. In the literature [3,20,21], many methods have been proposed and were all performed in the following order, the data partition phase, then the data distribution phase, followed by the data compression phase, and called the Send Followed Compress (SFC). These methods are all focused on sparse arrays based on the traditional matrix representation (TMR) [12].…”

Efficient Data Distribution Schemes for EKMR-Based Sparse Arrays on Distributed Memory Multicomputers

“…Ziantz et al [32] proposed a run-time optimization technique that was applied to sparse arrays for array distributions and off-processor data fetching to reduce the communication and the computation time. In their technique, they used the Block partition method with a binpacking algorithm to distribute a global sparse array to processors.…”

Data distribution schemes of sparse arrays on distributed memory multicomputers

“…In the data distribution phase, these local sparse arrays are distributed to processors. In the data compression phase, a local sparse array is compressed by data compression methods in order to obtain better performance for sparse array operations [7,15,16,18,21,23,26,30]. A data distribution scheme with this order is called the Send Followed Compress (SFC) scheme.…”