Tile Low Rank Cholesky Factorization for Climate/Weather Modeling Applications on Manycore Architectures

Abstract

Covariance matrices are ubiquitous in computational science and engineering. In particular, large covariance matrices arise from multivariate spatial data sets, for instance, in climate/weather modeling applications to improve prediction using statistical methods and spatial data. One of the most time-consuming computational steps consists in calculating the Cholesky factorization of the symmetric, positive-definite covariance matrix problem. The structure of such covariance matrices is also often data-sparse, in other words, effectively of low rank, though formally dense. While not typically globally of low rank, covariance matrices in which correlation decays with distance are nearly always hierarchically of low rank. While symmetry and positive definiteness should be, and nearly always are, exploited for performance purposes, exploiting low rank character in this context is very recent, and will be a key to solving these challenging problems at large-scale dimensions. The authors design a new and flexible tile row rank Cholesky factorization and propose a high performance implementation using OpenMP taskbased programming model on various leading-edge manycore architectures. Performance comparisons and memory footprint saving on up to 200K × 200K covariance matrix size show a gain of more than an order of magnitude for both metrics, against state-of-the-art open-source and vendor optimized numerical libraries, while preserving the numerical accuracy fidelity of the original model. This research represents an important milestone in enabling large-scale simulations for covariancebased scientific applications.

Sparse supernodal solver using block low-rank compression: Design, performance and analysis

“…Block Low-Rank compression has been investigated for dense matrices [18,19], and for sparse linear systems when using a multifrontal method [20,21]. Considering that these approaches are similar to the current study, a detailed comparison will be described in Section 6.…”

Exploiting Data Sparsity for Large-Scale Matrix Computations

“…More recently, with the emergence of asynchronous task-based programming models, these hierarchical low-rank matrix approximations algorithms have been revisited by flattening their recursions and exposing them to task-based runtime systems such as Intel Threading Building Blocks (Intel TBB) [22] and OpenMP [3]. While these dynamic runtimes permit to mitigate the overhead from the bus bandwidth saturation on single shared-memory nodes, they do not support distributed-memory systems.…”

“…This paper introduces the HiCMA library, the first implementation of taskbased tile low-rank Cholesky factorization on distributed-memory systems. Compared to the initial implementation on shared-memory environment [3] based on OpenMP, this paper uses instead the StarPU [9] dynamic runtime system to asynchronously schedule computational tasks across interconnected remote nodes. This highly productive association of task-based programming model with dynamic runtime systems permits to tackle in a systematic way advanced hardware systems by abstracting their complexity from the numerical library developers.…”

“…Fig. 3(b) in [3] sketches the TLR matrix after compressing on-the-fly each tile with a specific application-dependent fixed accuracy. This may result into low-rank tiles with non-uniform ranks to maintain the overall expected accuracy.…”

“…26 as shown in Fig. 3(a) in [3], allows to generate TLR matrices with uniform ranks across all low-rank tiles. This rough approximations may be of high interest for speeding up sparse preconditioners (e.g., incomplete Cholesky/LU factorizations) during iterative solvers, since important a priori assumptions can be made to optimize and improve parallel performance.…”