Parallel Two-Sided Matrix Reduction to Band Bidiagonal Form on Multicore Architectures

Abstract

Abstract. The objective of this paper is to extend, in the context of multicore architectures, the concepts of tile algorithms [Buttari et al., 2007] for Cholesky, LU, QR factorizations to the family of two-sided factorizations. In particular, the bidiagonal reduction of a general, dense matrix is very often used as a pre-processing step for calculating the Singular Value Decomposition. Furthermore, in the Top500 list of June 2008, 98% of the fastest parallel systems in the world were based on multicores. This confronts the scientific software community with both a daunting challenge and a unique opportunity. The challenge arises from the disturbing mismatch between the design of systems based on this new chip architecture -hundreds of thousands of nodes, a million or more cores, reduced bandwidth and memory available to cores -and the components of the traditional software stack, such as numerical libraries, on which scientific applications have relied for their accuracy and performance. The manycore trend has even more exacerbated the problem, and it becomes critical to efficiently integrate existing or new numerical linear algebra algorithms suitable for such hardware. By exploiting the concept of tile algorithms in the multicore environment (i.e., high level of parallelism with fine granularity and high performance data representation combined with a dynamic data driven execution), the band bidiagonal reduction presented here achieves 94 Gflop/s on a 12000 × 12000 matrix with 16 Intel Tigerton 2.4 GHz processors. The main drawback of the tile algorithms approach for the bidiagonal reduction is that the full reduction can not be obtained in one stage. Other methods have to be considered to further reduce the band matrix to the required form.

Comparative study of one-sided factorizations with multiple software packages on multi-core hardware

“…We may expect that these results generalize somewhat to other linear algebra algorithms and even any algorithm that can be expressed by a DAG of fine-grain tasks. Preliminary experiments using tile algorithms for two-sided transformations, i.e., the Hessenberg reduction [22] (first step for the standard eigenvalue problem) and the bidiagonal reduction [23] (first step for the singular value decomposition), show promising results.…”

Parallel reduction to condensed forms for symmetric eigenvalue problems using aggregated fine-grained and memory-aware kernels

“…The reduction to symmetric band tridiagonal form can be easily derived for the upper case. All the operations will be then based on the LQ factorization numerical kernels, as described in Ltaief et al [18]. Most of the kernels from the first stage are compute-intensive and rely on Level 3 BLAS operations (i.e., matrix-matrix multiplication) to achieve high performance.…”

Dense linear algebra solvers for multicore with GPU accelerators

“…The development of high performance DLA algorithms for homogeneous multicores has been successful in some cases, like the one-sided factorizations [4], and difficult for others, like the two-sided factorizations [5]. The situation is similar for GPUs -some algorithms map well, others are more challenging.…”