Regularization parameter estimation for large-scale Tikhonov regularization using a priori information

Renaut, Rosemary A.; Hntynková, Iveta; Mead, Jodi

doi:10.1016/j.csda.2009.05.026

Cited by 34 publications

(43 citation statements)

References 23 publications

(47 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although the proof of the result on the degrees of freedom for the underdetermined case m < n effectively follows the ideas introduced [16,22], the modification presented here provides a stronger result which can also strengthen the result for the overdetermined case, m ≥ n.…”

Section: Theoretical Developmentmentioning

confidence: 64%

“…The algorithm in [16] was presented for small scale problems in which one can use the singular value decomposition (SVD) [5] for matrix G when L = I or the generalized singular value decomposition (GSVD) [19] of the matrix pair [W d G; L]. For the large scale case an approach using the Golub-Kahan iterative bidiagonalization based on the LSQR algorithm [20,21] was presented in [22] along with the extension of the algorithm for the non-central destribution of P σ L (m Tik (σ L )), namely when m 0 is unknown but may be estimated from a set of measurements. In this paper the χ 2 principle is first extended to the estimation of σ L for underdetermined problems, specifically for the central χ 2 distribution with known m 0 , with the proof of the result in Section 2 and examples in Section 2.4.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Regularization parameter estimation for underdetermined problems by the χ ² principle with application to 2D focusing gravity inversion

Vatankhah¹,

Renaut²,

Ardestani³

2014

Inverse Problems

View full text Add to dashboard Cite

Abstract. The χ 2 -principle generalizes the Morozov discrepancy principle to the augmented residual of the Tikhonov regularized least squares problem. For weighting of the data fidelity by a known Gaussian noise distribution on the measured data and, when the stabilizing, or regularization, term is considered to be weighted by unknown inverse covariance information on the model parameters, the minimum of the Tikhonov functional becomes a random variable that follows a χ 2 -distribution with m + p − n degrees of freedom for the model matrix G of size m × n and regularizer L of size p × n.Here it is proved that the result holds for the underdetermined case, m < n provided that m + p ≥ n and that the null spaces of the operators do not intersect. A Newton root-finding algorithm is used to find the regularization parameter α which yields the optimal inverse covariance weighting in the case of a white noise assumption on the mapped model data. It is implemented for small-scale problems using the generalized singular value decomposition, or singular value decomposition when L = I. Numerical results verify the algorithm for the case of regularizers approximating zero to second order derivative approximations, contrasted with the methods of generalized cross validation and unbiased predictive risk estimation. The inversion of underdetermined 2D focusing gravity data produces models with non-smooth properties, for which typical implementations in this field use the iterative minimum support stabilizer and both regularizer and regularizing parameter are updated each iteration. For a simulated data set with noise, the regularization parameter estimation methods for underdetermined data sets are used in this iterative framework, also contrasted with the L-curve and the Morozov Discrepancy principle. These experiments demonstrate the efficiency and robustness of the χ 2 -principle in this context, moreover showing that the L-curve and Morozov Discrepancy Principle are outperformed in general by the three other techniques. Furthermore, the minimum support stabilizer is of general use for the χ 2 -principle when implemented without the desirable knowledge of a mean value of the model.

show abstract

Section: Theoretical Developmentmentioning

confidence: 64%

Section: Introductionmentioning

confidence: 99%

Regularization parameter estimation for underdetermined problems by the χ ² principle with application to 2D focusing gravity inversion

Vatankhah¹,

Renaut²,

Ardestani³

2014

Inverse Problems

View full text Add to dashboard Cite

show abstract

“…This is an extension of the scalar χ 2 method [21,22,23,31] which can be viewed as a regularization method. The new method amounts to solving multiple χ 2 tests to give an equal number of equations as the number of unknowns in the diagonal weighting matrix for data or parameter misfits.…”

Section: Discussionmentioning

confidence: 99%

“…This statistical interpretation of the weights in (3) and multipliers in (8) gives us a method for calculating them, and we term it the χ 2 method for parameter estimation and uncertainty quantification [21,22,23,31].…”

Section: Regularization and Constrained Optimizationmentioning

confidence: 99%

See 1 more Smart Citation

Discontinuous parameter estimates with least squares estimators

Mead

2013

Applied Mathematics and Computation

View full text Add to dashboard Cite

We discuss weighted least squares estimates of ill-conditioned linear inverse problems where weights are chosen to be inverse error covariance matrices. Least squares estimators are the maximum likelihood estimate for normally distributed data and parameters, but here we do not assume particular probability distributions. Weights for the estimator are found by ensuring its minimum follows a χ 2 distribution. Previous work with this approach has shown that the it is competitive with regularization methods such as the L-curve and Generalized Cross Validation (GCV) [23]. In this work we extend the method to find diagonal weighting matrices, rather than a scalar regularization parameter. Diagonal weighting matrices are advantageous because they give piecewise smooth least squares estimates and hence are a mechanism through which least squares can be used to estimate discontinuous parameters. This is explained by viewing least squares estimation as a constrained optimization problem. Results with diagonal weighting matrices are given for a benchmark discontinuous inverse problem from [15]. In addition, the method is used to estimate soil moisture from data collected in the Dry Creek Watershed near Boise, Idaho. Parameter estimates are found that combine two different types of measurements, and weighting matrices are found that incorporate uncertainty due to spatial variation so that the parameters can be used over larger scales than those that were measured.

show abstract

Krylov methods for inverse problems: Surveying classical, and introducing new, algorithmic approaches

Gazzola

Landman

2020

GAMM-Mitteilungen

View full text Add to dashboard Cite

Large-scale linear systems coming from suitable discretizations of linear inverse problems are challenging to solve. Indeed, since they are inherently ill-posed, appropriate regularization should be applied; since they are large-scale, well-established direct regularization methods (such as Tikhonov regularization) cannot often be straightforwardly employed, and iterative linear solvers should be exploited. Moreover, every regularization method crucially depends on the choice of one or more regularization parameters, which should be suitably tuned. The aim of this paper is twofold: (a) survey some well-established regularizing projection methods based on Krylov subspace methods (with a particular emphasis on methods based on the Golub-Kahan bidiagonalization algorithm), and the so-called hybrid approaches (which combine Tikhonov regularization and projection onto Krylov subspaces of increasing dimension); (b) introduce a new principled and adaptive algorithmic approach for regularization similar to specific instances of hybrid methods. In particular, the new strategy provides reliable parameter choice rules by leveraging the framework of bilevel optimization, and the links between Gauss quadrature and Golub-Kahan bidiagonalization. Numerical tests modeling inverse problems in imaging illustrate the performance of existing regularizing Krylov methods, and validate the new algorithms. K E Y W O R D S hybrid methods, imaging problems, Krylov subspace methods, large-scale linear inverse problems, regularization parameter choice rules, Tikhonov regularization 1 INTRODUCTION This paper considers linear, large-scale, discrete ill-posed problems of the form Ax true + e = b true + e = b , (1) where the matrix A ∈ R m×n represents a forward mapping that is ill-conditioned with ill-determined rank (ie, the singular values of A decay and cluster at zero without an evident gap between two consecutive ones), x true ∈ R n is the desired solution, and e ∈ R m is some unknown noise that affects the data b ∈ R m. Systems like (1) typically stem from the discretization of first-kind Fredholm integral equations, and model inverse problems arising in a variety of applications, such This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

show abstract

Regularization parameter estimation for large-scale Tikhonov regularization using a priori information

Cited by 34 publications

References 23 publications

Regularization parameter estimation for underdetermined problems by the χ ² principle with application to 2D focusing gravity inversion

Regularization parameter estimation for underdetermined problems by the χ ² principle with application to 2D focusing gravity inversion

Discontinuous parameter estimates with least squares estimators

Krylov methods for inverse problems: Surveying classical, and introducing new, algorithmic approaches

Contact Info

Product

Resources

About

Regularization parameter estimation for large-scale Tikhonov regularization using a priori information

Cited by 34 publications

References 23 publications

Regularization parameter estimation for underdetermined problems by the χ 2 principle with application to 2D focusing gravity inversion

Regularization parameter estimation for underdetermined problems by the χ 2 principle with application to 2D focusing gravity inversion

Discontinuous parameter estimates with least squares estimators

Krylov methods for inverse problems: Surveying classical, and introducing new, algorithmic approaches

Contact Info

Product

Resources

About

Regularization parameter estimation for underdetermined problems by the χ ² principle with application to 2D focusing gravity inversion

Regularization parameter estimation for underdetermined problems by the χ ² principle with application to 2D focusing gravity inversion