Probabilistic linear solvers: a unifying view

Abstract

Several recent works have developed a new, probabilistic interpretation for numerical algorithms solving linear systems in which the solution is inferred in a Bayesian framework, either directly or by inferring the unknown action of the matrix inverse. These approaches have typically focused on replicating the behavior of the conjugate gradient method as a prototypical iterative method. In this work surprisingly general conditions for equivalence of these disparate methods are presented. We also describe connections between probabilistic linear solvers and projection methods for linear systems, providing a probabilistic interpretation of a far more general class of iterative methods. In particular, this provides such an interpretation of the generalised minimum * Authors contributed equally. residual method. A probabilistic view of preconditioning is also introduced. These developments unify the literature on probabilistic linear solvers, and provide foundational connections to the literature on iterative solvers for linear systems.

Classical quadrature rules via Gaussian processes

“…Much foundational work on the rigorous definition of a Bayesian probabilistic numerical method has been done by Cockayne et al (2019a). Besides numerical integration, probabilistic methods have been developed for solving of ordinary (Skilling, 1992;Schober et al, 2018;Tronarp et al, 2019) and partial (Cockayne et al, 2017) differential equations and for numerical linear algebra (Hennig, 2015;Bartels et al, 2018;Cockayne et al, 2019b).…”

ProbNum: Probabilistic Numerics in Python

“…ProbNum implements iterative solvers, which either compute a belief over the (pseudo-)inverse of the matrix [7,8] or the solution directly [9,10]. While these perspectives differ conceptually, they are inherently connected [18]. belief update Ordinary Differential Equations ProbNum's stable implementation [11] of filtering-based ODE solvers and their variations [13,14] enables uncertainty calibration, step-size adaptation, dense output, event handling and posterior sampling [11,12].…”

Preconditioners for Krylov subspace methods: An overview

“…In particular [395] extends probabilistic interpretations of secant methods and mentions the potential for preconditioning the derived method, and [396] discusses in detail preconditioners for the authors' BayesCG method, which are thought of as priors. As outlined in [397], the conceptual difference is that in [395] a posterior is constructed on A −1 whereas in [396] a posterior is constructed on the solution x . The work [397] unifies the two approaches presented in [395,396], provides an interpretation of left‐ and right‐preconditioning, and extends the idea to the GMRES algorithm. Preconditioning and tensor methods: An important consideration when solving linear systems, for instance those arising from high‐dimensional PDEs, is that of data storage on a computer, which can become prohibitive unless specialized schemes are devised in order to combat this.…”

“…As outlined in [397], the conceptual difference is that in [395] a posterior is constructed on A −1 whereas in [396] a posterior is constructed on the solution x . The work [397] unifies the two approaches presented in [395,396], provides an interpretation of left‐ and right‐preconditioning, and extends the idea to the GMRES algorithm. Preconditioning and tensor methods: An important consideration when solving linear systems, for instance those arising from high‐dimensional PDEs, is that of data storage on a computer, which can become prohibitive unless specialized schemes are devised in order to combat this. One area of interest is that of utilizing low‐rank tensor decompositions alongside preconditioned solvers, for linear systems which possess some underlying tensor‐product structure.…”

“…As outlined in [397], the conceptual difference is that in [395] a posterior is constructed on A −1 whereas in [396] a posterior is constructed on the solution x . The work [397] unifies the two approaches presented in [395,396], provides an interpretation of left‐ and right‐preconditioning, and extends the idea to the GMRES algorithm.…”