Hierarchical algorithms on hierarchical architectures

Abstract

<jats:p>A traditional goal of algorithmic optimality, squeezing out flops, has been superseded by evolution in architecture. Flops no longer serve as a reasonable proxy for all aspects of complexity. Instead, algorithms must now squeeze memory, data transfers, and synchronizations, while extra flops on locally cached data represent only small costs in time and energy. Hierarchically low-rank matrices realize a rarely achieved combination of optimal storage complexity and high-computational intensity for a wide class of formally dense linear operators that arise in applications for which exascale computers are being constructed. They may be regarded as algebraic generalizations of the fast multipole method. Methods based on these hierarchical data structures and their simpler cousins, tile low-rank matrices, are well proportioned for early exascale computer architectures, which are provisioned for high processing power relative to memory capacity and memory bandwidth. They are ushering in a renaissance of computational linear algebra. A challenge is that emerging hardware architecture possesses hierarchies of its own that do not generally align with those of the algorithm. We describe modules of a software toolkit, hierarchical computations on manycore architectures, that illustrate these features and are intended as building blocks of applications, such as matrix-free higher-order methods in optimization and large-scale spatial statistics. Some modules of this open-source project have been adopted in the software libraries of major vendors.</jats:p> <jats:p>This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’.</jats:p>

Numerical algorithms for high-performance computational science

“…The original FMM method is kernel-dependent, but later on several kernel-independent methods have been proposed, kernel independent FMM, e.g., [70], hierarchical matrices, or H 2 matrices. For a discussion of hierarchical matrices see, e.g., [71], [72], [73], and [74].…”

Hierarchical matrix approximations for space-fractional diffusion equations

“…which produces the sum of a hierarchical matrix A of size N × N and a globally low rank matrix whose X and Y factors are of size N × k with k N . This operation can be efficiently implemented [30,54] by first adding the contributions of XY T to the various blocks of A at all levels, and recompressing the resulting sum algebraically as described earlier. The low rank update operation is a key routine for an operation that generates an explicit hierarchical matrix representation of an operator accessible only via matrix vector products.…”

“…Modern scientific workstations generally feature manycore GPU accelerators, and algorithms that do not effectively take advantage of these architectures are unlikely to be competitive for scientific and financial applications. Modern GPU architectures feature decreasing ratios of memory bandwidth to processing power, smaller amounts of fast memory per processing core, and substantial latencies for accessing data in deep memory [30]. Competitive algorithms must therefore be able to orchestrate their computations for effective execution in this environment.…”

Analysis of Parallelization Strategies in the context of Hierarchical Matrix Factorizations

“…• In our work we tackle multicore architectures. There exist recent research efforts to execute H-Matrices operations in distributed systems [74,116]. In fact, we have already developed a distributed memory implementation of the H-Chameleon library.…”

“…• En nuestro trabajo abordamos ejecuciones paralelas en arquitecturas multinúcleo. Hay estudios recientes que tratan de ejecutar operaciones sobre H-Matrices en sistemas distribuidos [74,116]. De hecho, nosotros hemos desarrollado una implementación de la biblioteca H-Chameleon para memoria distribuida.…”