Recursive Blocked Algorithms and Hybrid Data Structures for Dense Matrix Library Software

Abstract

Matrix computations are both fundamental and ubiquitous in computational science and its vast application areas. Along with the development of more advanced computer systems with complex memory hierarchies, there is a continuing demand for new algorithms and library software that efficiently utilize and adapt to new architecture features. This article reviews and details some of the recent advances made by applying the paradigm of recursion to dense matrix computations on today's memory-tiered computer systems. Recursion allows for efficient utilization of a memory hierarchy and generalizes existing fixed blocking by introducing automatic variable blocking that has the potential of matching every level of a deep memory hierarchy. Novel recursive blocked algorithms offer new ways to compute factorizations such as Cholesky and QR and to solve matrix equations. In fact, the whole gamut of existing dense linear algebra factorization is beginning to be reexamined in view of the recursive paradigm. Use of recursion has led to using new hybrid data structures and optimized superscalar kernels. The results we survey include new algorithms and library software implementations for level 3 kernels, matrix factorizations, and the solution of general systems of linear equations and several common matrix equations. The software implementations we survey are robust and show impressive performance on today's high performance computing systems.

Redesigning Triangular Dense Matrix Computations on GPUs

“…al [9] with packed and general data formats, and by Elmorth et. al [11] with recursive blocking. However, their formulations are designed for the hierarchical memory architecture of x86 multicore processors.…”

“…al [9], and Elmorth et. al [11]. In this paper, we describe a recursive formulation of the TRMM and TRSM kernels, which suits well the aggressively parallel many-core GPUs architecture.…”

An Algorithm-by-Blocks for SuperMatrix Band Cholesky Factorization

“…Since the early 1990s, various researchers [10,12,13,16] have proposed that matrices should be stored by blocks as opposed to the more customary columnmajor storage used in Fortran and row-major storage used in C. Doing so recursively is a generalization of that idea. The original reason was that by storing matrices contiguously a performance benefit would result.…”

Optimization of Triangular Matrix Functions in BLAS Library on Loongson2F

“…On optimization of linear algebra library, contemporary research focuses on an algorithmic level [2,3]. In a period of time that CPU reads one byte from memory, it can execute hundreds of instructions.…”