The regularizing effect of the Golub-Kahan iterative bidiagonalization and revealing the noise level in the data

Hnětýnková, Iveta; Plešinger, Martin; Strakoš, Zdeněk

doi:10.1007/s10543-009-0239-7

Cited by 50 publications

(78 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a CGtype method applied to the semidefinite linear system Ax = b or the normal equations system A T Ax = A T b, the CGLS algorithm has been studied; see [6,20,22] and the references therein. The LSQR algorithm [33], which is mathematically equivalent to CGLS, has attracted great attention, and is known to have regularizing effects and exhibits semi-convergence (see [20, p. 135], [22, p. 110], and also [5,19,23,32]): The iterates tend to be better and better approximations to the exact solution x true and their norms increase slowly and the residual norms decrease. In later stages, however, the noise e starts to deteriorate the iterates, so that they will start to diverge from x true and instead converge to the naive solution x naive , while their norms increase considerably and the residual norms stabilize.…”

Section: T Sv D K0mentioning

confidence: 99%

Some results on the regularization of LSQR for large-scale discrete ill-posed problems

Huang

Jia

2016

Sci. China Math.

View full text Add to dashboard Cite

Abstract. LSQR, a Lanczos bidiagonalization based Krylov subspace iterative method, and its mathematically equivalent CGLS applied to normal equations system, are commonly used for largescale discrete ill-posed problems. It is well known that LSQR and CGLS have regularizing effects, where the number of iterations plays the role of the regularization parameter. However, it has long been unknown whether the regularizing effects are good enough to find best possible regularized solutions. Here a best possible regularized solution means that it is at least as accurate as the best regularized solution obtained by the truncated singular value decomposition (TSVD) method. In this paper, we establish bounds for the distance between the k-dimensional Krylov subspace and the k-dimensional dominant right singular space. They show that the Krylov subspace captures the dominant right singular space better for severely and moderately ill-posed problems than for mildly ill-posed problems. Our general conclusions are that LSQR has better regularizing effects for the first two kinds of problems than for the third kind, and a hybrid LSQR with additional regularization is generally needed for mildly ill-posed problems. Exploiting the established bounds, we derive an estimate for the accuracy of the rank k approximation generated by Lanczos bidiagonalization. Numerical experiments illustrate that the regularizing effects of LSQR are good enough to compute best possible regularized solutions for severely and moderately ill-posed problems, stronger than our theory predicts, but they are not for mildly ill-posed problems and additional regularization is needed.

show abstract

Section: T Sv D K0mentioning

confidence: 99%

Some results on the regularization of LSQR for large-scale discrete ill-posed problems

Huang

Jia

2016

Sci. China Math.

View full text Add to dashboard Cite

show abstract

“…Hnětynková et al [30] recently proposed to use the connection between Lanczos bidiagonalization (referred to as Golub-Kahan bidiagonalization) and Gauss quadrature to estimate the norm of the error e in b. We refer to this approach as the quadrature method.…”

Section: The Quadrature Methodsmentioning

confidence: 99%

“…Hnětynková et al [30] use this connection between Lanczos bidiagonalization and Gauss quadrature to derive a method for detecting the amount of "noise" in the vector b. Introduce the singular value decomposition…”

Section: The Quadrature Methodsmentioning

confidence: 99%

“…Hnětynková et al [30] say that stagnation occurs at step k, and use the iterate x k as an approximation ofx.…”

Section: The Quadrature Methodsmentioning

confidence: 99%

“…in (19) be numerically orthonormal for the good performance of this regularization parameter choice rule. To secure numerical orthogonality, Hnětynková et al [30] carry out double reorthogonalization; the software they provide for bidiagonalization is used in the numerical illustrations presented in Section 4.…”

Section: The Quadrature Methodsmentioning

confidence: 99%

See 2 more Smart Citations

Old and new parameter choice rules for discrete ill-posed problems

2012

View full text Add to dashboard Cite

Krylov methods for inverse problems: Surveying classical, and introducing new, algorithmic approaches

Gazzola

Landman

2020

GAMM-Mitteilungen

View full text Add to dashboard Cite

Large-scale linear systems coming from suitable discretizations of linear inverse problems are challenging to solve. Indeed, since they are inherently ill-posed, appropriate regularization should be applied; since they are large-scale, well-established direct regularization methods (such as Tikhonov regularization) cannot often be straightforwardly employed, and iterative linear solvers should be exploited. Moreover, every regularization method crucially depends on the choice of one or more regularization parameters, which should be suitably tuned. The aim of this paper is twofold: (a) survey some well-established regularizing projection methods based on Krylov subspace methods (with a particular emphasis on methods based on the Golub-Kahan bidiagonalization algorithm), and the so-called hybrid approaches (which combine Tikhonov regularization and projection onto Krylov subspaces of increasing dimension); (b) introduce a new principled and adaptive algorithmic approach for regularization similar to specific instances of hybrid methods. In particular, the new strategy provides reliable parameter choice rules by leveraging the framework of bilevel optimization, and the links between Gauss quadrature and Golub-Kahan bidiagonalization. Numerical tests modeling inverse problems in imaging illustrate the performance of existing regularizing Krylov methods, and validate the new algorithms. K E Y W O R D S hybrid methods, imaging problems, Krylov subspace methods, large-scale linear inverse problems, regularization parameter choice rules, Tikhonov regularization 1 INTRODUCTION This paper considers linear, large-scale, discrete ill-posed problems of the form Ax true + e = b true + e = b , (1) where the matrix A ∈ R m×n represents a forward mapping that is ill-conditioned with ill-determined rank (ie, the singular values of A decay and cluster at zero without an evident gap between two consecutive ones), x true ∈ R n is the desired solution, and e ∈ R m is some unknown noise that affects the data b ∈ R m. Systems like (1) typically stem from the discretization of first-kind Fredholm integral equations, and model inverse problems arising in a variety of applications, such This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

show abstract

The regularizing effect of the Golub-Kahan iterative bidiagonalization and revealing the noise level in the data

Cited by 50 publications

References 46 publications

Some results on the regularization of LSQR for large-scale discrete ill-posed problems

Some results on the regularization of LSQR for large-scale discrete ill-posed problems

Old and new parameter choice rules for discrete ill-posed problems

Krylov methods for inverse problems: Surveying classical, and introducing new, algorithmic approaches

Contact Info

Product

Resources

About