Efficient dense matrix‐vector multiplication on GPU

Abstract

Given that the dense matrix-vector multiplication (Ax or A T x) is of great importance in scientific computations, how to accelerate it is investigated on the graphics processing unit (GPU) in this paper. We present a warp-based implementation of Ax on the GPU, called GEMV-Adaptive, and a thread-based implementation of A T x on the GPU, called GEMV-T-Adaptive. For our proposed GEMV-Adaptive and GEMV-T-Adaptive, there are the following novelties: (1) an adaptive warp allocation strategy for GEMV-Adaptive is proposed to assign the optimal warp number for each matrix row, (2) an adaptive thread allocation strategy for GEMV-T-Adaptive is designed to assign the optimal thread number to each matrix row, and (3) several optimization schemes are formulated. Experimental results show that the proposed GEMV-Adaptive and GEMV-T-Adaptive mitigate the performance fluctuations of the implementations in the CUBLAS library, always have high performance, and outperform the most recently proposed GEMV and GEMV-T kernels by Gao et al, respectively, for all test matrices. KEYWORDS CUDA, dense matrix-vector multiplication, GPU INTRODUCTIONThe dense matrix-vector multiplication routine performs one ofwhere A ∈ R m×n is a dense matrix, and y and x are vectors. It has proven to be of particular importance in computational science and has been successfully applied in various fields. 1-7 As the matrix size increases in the practical problems, parallel computing is required to efficiently improve the performance of the dense matrix-vector multiplication.Processing big data by using graphics processing units (GPUs) has drawn much attention over the recent years. Following the introduction of the compute unified device architecture (CUDA), a programming model that supports the joint CPU/GPU execution of applications, by NVIDIA 8 in 2007,GPUs have become strong competitors as general-purpose parallel programming systems.Researchers have recently developed suitable and flexible dense matrix-vector multiplication algorithms on the GPU architecture. 9-14 The representative of them is KBLAS by Abdelfattah et al. 14 KBLAS is an optimized library for dense MVM kernels on GPUs and can efficiently run on various GPU architectures while avoiding code rewriting and retaining compliance with the standard BLAS API. Experimental results show that the KBLAS performance either matches or outperforms existing state-of-the-art open-source and commercial implementations (eg, NVIDIA's standard BLAS implementation CUBLAS, 15 MAGMABLAS, 16 and CULA 17 ) on different matrix sizes. A subset of KBLAS high-performance kernels has been integrated into CUBLAS for larger dissemination, starting with version 6.0. 18 Concurrency Computat Pract Exper. 2018;30:e4705. wileyonlinelibrary.com/journal/cpe

Fast Kronecker Matrix-Matrix Multiplication on GPUs

“…Optimizing Small and Skinny Matmul Many works has optimized Matrix Multiplication and Matrix-Vector Multiplication computations on small and skinny matrices on GPUs [6,19,39]. He et al [19] proposes an optimal warp allocation strategy for matrix-vector multiplication. KBLAS [6] uses double-buffering to overlap data motion with computation to optimize matrix-vector multiplication.…”

Optimization of Clutter Simulation Based on GPU

“…when p is different, its calculation effect is quite different. The same test was performed on the gemv function, and the results show that the performance of gemv in different dimensions is also quite different [18]. when using the Sunway TaihuLight supercomputer to perform the convolution operation using the gemm matrix multiplication method, it was found that the calculation efficiency was low due to the large difference between the convolution array and the convolution kernel.…”

A new diagonal storage for efficient implementation of sparse matrix–vector multiplication on graphics processing unit

“…Processing big data by using GPUs has drawn much attention over the recent years. Following the introduction of the compute unified device architecture (CUDA), a programming model that supports the joint CPU/GPU execution of applications, by NVIDIA in 2007, 9 GPUs have become strong competitors as general‐purpose parallel programming systems, and have been increasingly used as tools for high‐performance computation in many fields 10–17 …”