Smat

Abstract

Sparse Matrix Vector multiplication (SpMV) is an important kernel in both traditional high performance computing and emerging data-intensive applications. By far, SpMV libraries are optimized by either application-specific or architecture-specific approaches, making the libraries become too complicated to be used extensively in real applications. In this work we develop a Sparse Matrixvector multiplication Auto-Tuning system (SMAT) to bridge the gap between specific optimizations and general-purpose usage. S-MAT provides users with a unified programming interface in compressed sparse row (CSR) format and automatically determines the optimal format and implementation for any input sparse matrix at runtime. For this purpose, SMAT leverages a learning model, which is generated in an off-line stage by a machine learning method with a training set of more than 2000 matrices from the UF sparse matrix collection, to quickly predict the best combination of the matrix feature parameters. Our experiments show that SMAT achieves impressive performance of up to 51GFLOPS in single-precision and 37GFLOPS in double-precision on mainstream x86 multi-core processors, which are both more than 3 times faster than the Intel MKL library. We also demonstrate its adaptability in an algebraic multigrid solver from Hypre library with above 20% performance improvement reported.

Optimizing Sparse Linear Algebra Through Automatic Format Selection and Machine Learning

“…Furthermore, choosing the optimal format by running the available options first can result in significant overheads. ML and Artificial Intelligence (AI) have been successful in various optimization tasks ranging from code optimization to model selection [18], including the task of selecting the optimal sparse matrix storage format [13], [16]. Adopting a ML model has the potential to offer an accurate and low-overhead solution to the problem of automatic format selection, eliminating any requirement for manual format selection input from the user enabling applications to remain optimal across the different types of hardware and sparsity patterns for any operation of interest.…”

“…Similarly, Sedeghati et al [26] used a Decision Tree based classifier to choose from five available formats resulting in 81% accuracy on GPUs. On the other hand, Li et al [13] used an input adaptive SpMV auto-tuner based on ruleset classification that maintains a confidence value for each test sample. If the prediction of the classifier is below the defined threshold, the auto-tuner selects the optimal format and SpMV kernel otherwise switches to a run-first approach to make the decision, reporting accuracy of up to 85%.…”

“…Even though it is crucial to enable some dynamic switching mechanism to be able to change formats at runtime, it is also important to automate the process of selecting the optimal format. To facilitate the selection of the optimal format for a given matrix, target architecture and operation, auto-tuners have been developed (such as [9], [13], [15], [16]) with a focus on selecting the optimal format for the Sparse Matrix-Vector Multiplication (SpMV), the operation that often dominates the runtime of computing the solution to linear systems. These contributions demonstrated the impact of optimizing the performance of iterative solvers through automatic format selection on single node multi-and many-core systems.…”

PASTA: a parallel sparse tensor algorithm benchmark suite

“…is phenomenon has been observed from sparse matrices, where di erent sparse formats behave quite di erently on diverse input matrices [87,119,139,163]. As mentioned in the work [12], an outdated algorithm cannot well re ect the current status of an application.…”

A parallel sparse tensor benchmark suite on CPUs and GPUs

“…Our benchmark suite consists of a set of reference implementations from various tensor applications, each of which show di erent computational behavior. Much like two-dimensional sparse matrices, the data layout, or the data structure used to hold a sparse tensor, has a signi cant impact on performance and storage [43,54]. It also has a signi cant impact on how the control ow for a given operation must be executed and its memory footprints.…”