Auto-Tuning of Thread Assignment for Matrix-Vector Multiplication on GPUs

Abstract

SUMMARYModern GPUs have evolved to become a more general processor capable of executing scientific and engineering computations. It provides a highly parallel computing environment due to its large number of computing cores, which are suitable for numerous data parallel arithmetic computations, particularly linear algebra operations. The matrixvector multiplication is one of the most important dense linear algebraic operations. It is applied to a diverse set of applications in many fields and must therefore be fully optimized to achieve a high-performance. In this paper, we proposed a novel auto-tuning method for matrix-vector multiplication on GPUs, where the number of assigned threads that are used to compute one element of the result vector can be auto-tuned according to the size of matrix. On the Nvidia's GPU GTX 650 with the most recent Kepler architecture, we developed an auto-tuner that can automatically select the optimal number of assigned threads for calculation. Based on the auto-tuner's result, we developed a versatile generic matrix-vector multiplication kernel with the CUDA programming model. A series of experiments on different shapes and sizes of matrices were conducted for comparing the performance of our kernel with that of the kernels from CUBLAS 5.0, MAGMA 1.3 and a warp method. The experiments results show that the performance of our matrix-vector multiplication kernel is close to the optimal behavior with increasing of the size of the matrix and has very little dependency on the shape of the matrix, which is a significant improvement compared to the other three kernels that exhibit unstable performance behavior for different shapes of matrices.

Efficient Parallel Implementation of Active Appearance Model Fitting Algorithm on GPU

“…Thus, when the matrix is very wide, each thread conducts a large number of calculations and the performance will observably reduce. Therefore, we designed a novel autotuning method for matrix-vector multiplication on GPUs, where the number of threads used to compute one element of the result vector can be autotuned according to the matrix size [ 29 ]. For very wide matrices, thousands of threads are used to compute one element of the result vector.…”

Padding Free Bank Conflict Resolution for CUDA-Based Matrix Transpose Algorithm

“…GPU on the other hand is capable of executing more GFLOPS than normal CPU. It provides a highly parallel computing environment suitable for numerous data parallel arithmetic computations such as dense linear algebraic operations 13 . However, the only restriction in earlier GPU version lies in its lack of support for IEEE FP Standards 12 .…”