On the Parallel I/O Optimality of Linear Algebra Kernels: Near-Optimal LU Factorization

Abstract

Dense linear algebra kernels, such as linear solvers or tensor contractions, are fundamental components of many scientific computing applications. In this work we present a novel method of deriving parallel I/O lower bounds for this broad family of programs. Based on the 𝑋 -Partitioning abstraction, our method explicitly captures inter-statement dependencies. Applying our analysis to LU factorization, we derive COnf LUX, an LU algorithm with the parallel I/O cost of 𝑁 3 /(𝑃 √ 𝑀) communicated elements per processor -only 1/3× over our established lower bound. We evaluate COnf LUX on various problem sizes, demonstrating empirical results that match our theoretical analysis, communicating asymptotically less than Cray ScaLAPACK or SLATE, and outperforming the asymptoticallyoptimal CANDMC library. Running on 1,024 nodes of Piz Daint, COnf LUX communicates 1.6× less than the second-best implementation and is expected to communicate 2.1× less on a full-scale run on Summit.