Anatomy of high-performance matrix multiplication

Abstract

We present the basic principles that underlie the high-performance implementation of the matrix-matrix multiplication that is part of the widely used GotoBLAS library. Design decisions are justified by successively refining a model of architectures with multilevel memories. A simple but effective algorithm for executing this operation results. Implementations on a broad selection of architectures are shown to achieve near-peak performance.

Evaluation and tuning of the Level 3 CUBLAS for graphics processors

“…Following the characterization of the matrix multiplication in [5], we next analyze the performance of this operation when one of the matrix dimensions (m, n, or k) is small with respect to the other two. This gives us three different kernels: SGEPM (m is small), SGEMP (n is small), and SGEPP (k is small).…”

A comparison of high-order time integrators for thermal convection in rotating spherical shells

“…The FFT routines used to evaluate the non-linear terms are those of the FFTW3 library [25]. The action of the linear operators required by the linear solvers, the computation of the non-linear terms, and the Legendre transforms have been implemented as matrix-matrix products using the GotoBLAS library [26] to increase the efficiency of the codes. The block structure of some of the matrices is used to minimize the number of operations.…”

“…At first sight it could seem from (26)(27), and the values of the factors given above, that the computational cost of the Q-implicit method is much higher than for the others, and that it does not depend on the order. However, as the order increases, the predictions of the solution at the end of each step, based on extrapolation using the order of the integrator, are better, and then the number of iterations N GMR to solve the linear system during the corrections is lower.…”

Deriving dense linear algebra libraries

“…The commands in the body of the loop map are implemented as calls to Basic Linear Algebra Subprograms (BLAS) [LHKK79,DDCHH88,DDCHD90], an interface to commonly encountered linear algebra operations, as well as other routines supported by libflame, which themselves call BLAS operations. As part of our project, we have derived a full library of these operations, but for this experiment we are depending on optimized implementations provided by the GotoBLAS2 implementation [GvdG08b,GvdG08a]. For the blocked algorithms, a block size of 128 was used.…”