Exploiting variable precision in GMRES

Gratton, Serge; Simon, Ehouarn; Titley-Peloquin, David; Toint, Philippe

doi:10.48550/arxiv.1907.10550

Cited by 3 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, Gratton, Simon, Titley-Peloquin and Toint (2019) prove that the orthonormalization of the Krylov basis can also be performed inexactly. This observation is leveraged by Aliaga et al (2020), who propose to store the Krylov basis in lower precision.…”

Section: Mixed Precision Gmresmentioning

confidence: 90%

Mixed precision algorithms in numerical linear algebra

Higham¹,

Mary²

2022

Acta Numerica

View full text Add to dashboard Cite

Today’s floating-point arithmetic landscape is broader than ever. While scientific computing has traditionally used single precision and double precision floating-point arithmetics, half precision is increasingly available in hardware and quadruple precision is supported in software. Lower precision arithmetic brings increased speed and reduced communication and energy costs, but it produces results of correspondingly low accuracy. Higher precisions are more expensive but can potentially provide great benefits, even if used sparingly. A variety of mixed precision algorithms have been developed that combine the superior performance of lower precisions with the better accuracy of higher precisions. Some of these algorithms aim to provide results of the same quality as algorithms running in a fixed precision but at a much lower cost; others use a little higher precision to improve the accuracy of an algorithm. This survey treats a broad range of mixed precision algorithms in numerical linear algebra, both direct and iterative, for problems including matrix multiplication, matrix factorization, linear systems, least squares, eigenvalue decomposition and singular value decomposition. We identify key algorithmic ideas, such as iterative refinement, adapting the precision to the data, and exploiting mixed precision block fused multiply–add operations. We also describe the possible performance benefits and explain what is known about the numerical stability of the algorithms. This survey should be useful to a wide community of researchers and practitioners who wish to develop or benefit from mixed precision numerical linear algebra algorithms.

show abstract

Section: Mixed Precision Gmresmentioning

confidence: 90%

Mixed precision algorithms in numerical linear algebra

Higham¹,

Mary²

2022

Acta Numerica

View full text Add to dashboard Cite

show abstract

“…Many new mixed-precision algorithms are being developed by the numerical linear algebra community, among which algorithms for matrix factorization [12,15,48,66,67], iterative refinement [11,18,19], and Krylov subspace methods [10,28]. For an overview of recent developments in mixed-precision computing we refer to this excellent community review [1].…”

Section: Introductionmentioning

confidence: 99%

“…The development of preconditioned iterative linear solvers is an active field of investigation due to the complications arising with loss of orthogonality of the Arnoldi/Lanczos vectors [14,52]. However, some new fascinating results have been obtained for mixed-precision GMRES [28], and flexible GMRES [10]. Mixed-precision multigrid solvers based on iterative refinement have also been developed [49,57].…”

Section: Introductionmentioning

confidence: 99%

Mixed-precision explicit stabilized Runge-Kutta methods for single- and multi-scale differential equations

Croci,

de Souza

2021

Preprint

View full text Add to dashboard Cite

Mixed-precision algorithms combine low-and high-precision computations in order to benefit from the performance gains of reduced-precision without sacrificing accuracy. In this work, we design mixedprecision Runge-Kutta-Chebyshev (RKC) methods, where high precision is used for accuracy, and low precision for stability. Generally speaking, RKC methods are low-order explicit schemes with a stability domain growing quadratically with the number of function evaluations. For this reason, most of the computational effort is spent on stability rather than accuracy purposes. In this paper, we show that a naïve mixed-precision implementation of any Runge-Kutta scheme can harm the convergence order of the method and limit its accuracy, and we introduce a new class of mixed-precision RKC schemes that are instead unaffected by this limiting behaviour. We present three mixed-precision schemes: a first-and a second-order RKC method, and a first-order multirate RKC scheme for multiscale problems. These schemes perform only the few function evaluations needed for accuracy (1 or 2 for first-and second-order methods respectively) in high precision, while the rest are performed in low precision. We prove that while these methods are essentially as cheap as their fully low-precision equivalent, they retain the convergence order of their high-precision counterpart. Indeed, numerical experiments confirm that these schemes are as accurate as the corresponding high-precision method.

show abstract

“…The numerical properties of GMRES were analyzed in [8,13,22,24]. The usage of variable (or multi) precision arithmetic for Krylov methods, and in particular GMRES, was discussed in [9,12,27,30]. In the present article we chose the GMRES method as a practical application of the RGS algorithm.…”

Section: Introductionmentioning

confidence: 99%

Randomized Gram-Schmidt process with application to GMRES

Balabanov¹,

Grigori²

2020

Preprint

View full text Add to dashboard Cite

A randomized Gram-Schmidt algorithm is developed for orthonormalization of high-dimensional vectors or QR factorization. The proposed process can be less computationally expensive than the classical Gram-Schmidt process while being at least as numerically stable as the modified Gram-Schmidt process. Our approach is based on random sketching, which is a dimension reduction technique consisting in estimation of inner products of high-dimensional vectors by inner products of their small efficiently-computable random projections, so-called sketches. This allows to perform the projection step in Gram-Schmidt process on sketches rather than high-dimensional vectors with a minor computational cost. This also provides an ability to efficiently certify the output.The proposed Gram-Schmidt algorithm can provide computational cost reduction in any architecture. The benefit of random sketching can be amplified by exploiting multi-precision arithmetic. We provide stability analysis for multi-precision model with coarse unit roundoff for standard highdimensional operations. Numerical stability is proven for the unit roundoff independent of the (high) dimension of the problem.The proposed Gram-Schmidt process can be applied to Arnoldi iteration and result in new Krylov subspace methods for solving high-dimensional systems of equations or eigenvalue problems. Among them we chose randomized GMRES method as a practical application of the methodology.

show abstract

Exploiting variable precision in GMRES

Cited by 3 publications

References 15 publications

Mixed precision algorithms in numerical linear algebra

Mixed precision algorithms in numerical linear algebra

Mixed-precision explicit stabilized Runge-Kutta methods for single- and multi-scale differential equations

Randomized Gram-Schmidt process with application to GMRES

Contact Info

Product

Resources

About