Leveraging PaRSEC Runtime Support to Tackle Challenging 3D Data-Sparse Matrix Problems

Abstract

The task-based programming model associated with dynamic runtime systems has gained popularity for challenging problems because of workload imbalance, heterogeneous resources, or extreme concurrency. During the last decade, lowrank matrix approximations, where the main idea consists of exploiting data sparsity typically by compressing off-diagonal tiles up to an application-specific accuracy threshold, have been adopted to address the curse of dimensionality at extreme scale. In this paper, we create a bridge between the runtime and the linear algebra by communicating knowledge of the data sparsity to the runtime. We design and implement this synergistic approach with high user productivity in mind, in the context of the PaRSEC runtime system and the HiCMA numerical library. This requires to extend PaRSEC with new features to integrate rank information into the dataflow so that proper decisions can be taken at runtime. We focus on the tile low-rank (TLR) Cholesky factorization for solving 3D data-sparse covariance matrix problems arising in environmental applications. In particular, we employ the 3D exponential model of Mateŕn matrix kernel, which exhibits challenging nonuniform high ranks in off-diagonal tiles. We first provide a dynamic data structure management driven by a performance model to reduce extra floating-point operations. Next, we optimize the memory footprint of the application by relying on a dynamic memory allocator, and supported by a rank-aware data distribution to cope with the workload imbalance. Finally, we expose further parallelism using kernel recursive formulations to shorten the critical path. Our resulting high-performance implementation outperforms existing data-sparse TLR Cholesky factorization by up to 7-fold on a large-scale distributed-memory system, while minimizing the memory footprint up to a 44-fold factor. This multidisciplinary work highlights the need to empower runtime systems beyond their original duty of task scheduling for servicing next-generation low-rank matrix algebra libraries.

High performance computing seismic redatuming by inversion with algebraic compression and multiple precisions

“…Tile Low-Rank (TLR) matrix approximation (Amestoy et al, 2015); (Akbudak et al, 2017) simplifies the representative structure of the compressed data, making it amenable to supporting complex matrix operations on challenging hardware environments (Charara et al, 2018); (Keyes et al, 2020). TLR is therefore an excellent compromise between mathematical optimality and implementation complexity and has lately managed to penetrate a wide range of large-scale scientific applications (Abdulah et al, 2018); (Al-Harthi et al, 2020); (Cao et al, 2020); (Ltaief et al, 2021); (Hong et al, 2021); (Cao et al, 2021); Alomairy et al (2022), including geophysical processing (Hong et al, 2021); (Ltaief et al, 2023b,a). Compression capabilities of H-matrices in the context of seismic have been also reported in Jumah and Herrmann (2012).…”

Reshaping Geostatistical Modeling and Prediction for Extreme-Scale Environmental Applications

“…PaRSEC allows us to focus on algorithmic features and program in a style independent of the data distribution to reach unprecedented levels of efficiency for solving extreme-scale linear algebra matrix operations [25]. This effort is part of a larger ongoing research collaboration, as earlier described in [12], [32], [34], [35]. We further extend this work to seamlessly combine MP+Dense/TLR Cholesky 1) When to use MP: MP selection may be based in a brute force way on a band structure, as shown in Fig.…”

“…3(b) with a band of three tiles. This runtime decision has been previously studied on two largescale HPC systems, i.e., full-scale Shaheen II [32], [34], [35] and Fugaku [36]. However, the decision therein did not take into account the precision of each tile as we address it here.…”

“…ExaGeoStat has been ported to most of the major commercial hardware architectures of the Top10 supercomputers [12], [32], [35], including IBM+NVIDIA (Summit) and AMD (Hawk). In this paper, we run and validate our accuracy experiments on Shaheen II, a Cray XC40 supercomputer with 6,174 nodes composed of two-socket 16core Intel Haswell (AVX2) processor and 128GB of main memory, using the Cray Aries network interconnect.…”