Experiences in autotuning matrix multiplication for energy minimization on GPUs

Abstract

In this paper, we report extensive results and analysis of autotuning the computationally intensive graphics processing units kernel for dense matrix-matrix multiplication in double precision. In contrast to traditional autotuning and/or optimization for runtime performance only, we also take the energy efficiency into account. For kernels achieving equal performance, we show significant differences in their energy balance. We also identify the memory throughput as the most influential metric that trades off performance and energy efficiency. As a result, the performance optimal case ends up not being the most efficient kernel in overall resource use.The tunable GEMM implementation, called the stencil, is completely parametrized using the pyexpander preprocessor [34]. Figure 2 shows the five dimensions of tiling in the GEMM stencil. Each Recently, metrics accounting for performance and energy efficiency have become popular [39]. This approach is generally designed as a tradeoff between energy usage e and runtime t using a weight˛in the objective function to minimize M.t; e/ D˛ t C .1 ˛/ e:(2) Figure 9. Classification of the generated kernels with respect to the values of different metrics above (red) or below (green) average along with the achieved runtime performance and energy efficiency.

Massively Parallel Automated Software Tuning

“…1 Consider the architecture of the number one system on the list, the Summit supercomputer at the Oak Ridge Leadership Computing Facility (OLCF). 2 Summit contains three NVIDIA V100 GPUs per each POWER9 CPU. The peak double-precision floating-point performance of the CPU is 22 (cores) × 24.56 GF LOPS = 540.32 GF LOPS.…”

“…While in the previous sections we used the simplistic gemm kernel from the CUDA Programming Guide, here we are using a highly parametrized gemm kernel developed in the course of our past autotuning efforts [2,9,10]. It is based on a fairly standard approach, where values of C are accumulated in registers, while values of A and B are streamed through shared memory in thin stripes.…”

“…A is streamed through shared memory in stripes of size blk_m × blk_k and B is streamed in stripes of size blk_k × blk_n. For the lack of space, we refer the readers to the literature for more details [2,9,10].…”

Toward Exascale Computing Systems: An Energy Efficient Massive Parallel Computational Model

“…4. Based on performance and energy efficiency, a tradeoff between the two metrics [47] can be determined as follows:…”

Novel HPC techniques to batch execution of many variable size BLAS computations on GPUs

“…An important aspect of the proposed design is the reliance on the matrix-matrix multiplications (GEMM) kernel in almost all BLAS routines. This approach not only reduces the code base required for the development, but also achieves high performance for all routines, thanks to the GEMM kernel being a common target for continuing research, development, and tuning [21] [13] [3]. In fact, any performance improvement done to the GEMM kernel would automatically propagate to almost all other routines in Level-3 BLAS.…”