Performance modeling of the sparse matrix–vector product via convolutional neural networks

Abstract

Modeling the execution time of the Sparse Matrix-Vector multiplication (SpMV) on a current CPU architecture is especially complex due to i) irregular memory accesses; ii) indirect memory referencing; and iii) low arithmetic intensity. While analytical models may yield accurate estimates for the total number of cache hits/misses, they often fail to predict accurately the total execution time. In this paper, we depart from the analytic approach to instead leverage Convolutional Neural Networks (CNNs) in order to provide an effective estimation of the performance of the SpMV operation. For this purpose, we present a high-level abstraction of the sparsity pattern of the problem matrix and propose a blockwise strategy to feed the CNN models by blocks of non-zero elements. The experimental evaluation on a representative subset of the matrices from the SuiteSparse Matrix collection demonstrates the robustness of the CNN models for predicting the SpMV performance on an Intel Haswell core. Furthermore, we show how to generalize the network models to other target architectures to estimate the performance of SpMV on an ARM A57 core.

Convolutional neural nets for estimating the run time and energy consumption of the sparse matrix-vector product

“…Similarly, 80% of the training dataset was utilized only for training, and the remaining 20% was used for validation, to guide the training process. In Barreda et al (2020c) we showed that the trained models for the time metric using this dataset provide an appropriate generalization power. Using too much training data could lead to a model which overfits the problem.…”

“…The strategy of partitioning the vpos 0 array into blocks forces us to implement a blockwise version of the classic CSR-based SPMV Algorithm 1 to generate the training dataset for the CNNs, in which each block of vpos 0 is labeled with its corresponding ratios of execution time and energy consumption (total, package, and DRAM) per nonzero element. In a previous work (Barreda et al, 2020c), we analyzed the impact of the block size in the time predictions and concluded that the proposed network architectures deliver accurate results for small block sizes. The reason is that, in general, small blocks reflect a small set of sparsity patterns which, in turn, can be better captured by the CNN filters.…”

“…Specifically, Convolutional Neural Networks (CNNs) (Gu et al, 2018) may provide a powerful means to capture spatial and temporal dependencies using abstract representations of the sparse matrices involved in the SPMV through a set of convolutional filters. In this paper, we extend the work in (Barreda et al, 2020c) with the design of a more accurate execution time model and a new energy consumption model using CNNs as base algorithms. In particular, our paper makes the following contributions:…”

“…We study the estimations of these metrics considering different operating frequencies of the Intel Xeon Haswell core. We leverage the model migration technique presented in (Barreda et al, 2020c) for the different energy metrics and frequencies studied. We propose a blockwise realization to make the CNN model architecture independent of the sparse matrix dimension input blocks.…”

Selecting optimal SpMV realizations for GPUs via machine learning

“…Other interesting efforts related with this topic are the article by Williams et al (2007) where the authors study some optimization techniques for the SpMV kernel over several hardware platforms; the proposal by Erguiz et al (2017) with advances over the automatic selection of different sparse triangular linear solvers on GPU; and the work by Barreda et al (2020) which offers a performance modeling of the SpMV kernel via convolutional neural networks with ARM as the target hardware platform.…”