Convergence of Heuristic Parameter Choice Rules for Convex Tikhonov Regularization

Kindermann, Stefan; Raik, Kemal

doi:10.1137/19m1263066

Cited by 18 publications

(26 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In particular, this yields a rather simple explanation why heuristic parameter choice rules often do well in practice. A more detailed investigation and a relation to the theory of heuristic parameter choice rules (see, e.g., [24,25,26]) is left as future work.…”

Section: A New Methods and Comparisonmentioning

confidence: 99%

Estimating solution smoothness and data noise with Tikhonov regularization

Gerth¹,

Ramlau²

2020

Preprint

View full text Add to dashboard Cite

A main drawback of classical Tikhonov regularization is that often the parameters required to apply theoretical results, e.g., the smoothness of the sought-after solution and the noise level, are unknown in practice. In this paper we investigate in new detail the residuals in Tikhonov regularization viewed as functions of the regularization parameter. We show that the residual carries, with some restrictions, the information on both the unknown solution and the noise level. By calculating approximate solutions for a large range of regularization parameters, we can extract both parameters from the residual given only one set of noisy data and the forward operator. The smoothness in the residual allows to revisit parameter choice rules and relate apriori, a-posteriori, and heuristic rules in a novel way that blurs the lines between the classical division of the parameter choice rules. All results are accompanied by numerical experiments.

show abstract

Section: A New Methods and Comparisonmentioning

confidence: 99%

Estimating solution smoothness and data noise with Tikhonov regularization

Gerth¹,

Ramlau²

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…This is because solving a sequence of quadratic programs is a tough problem, and it is also sometimes not clear how to choose an appropriate sequence of regularization parameters {µ k } → 0 + . To be more precise, choosing the regularization parameter in Tikhonov regularization is still an open research line [8,16].…”

Section: Minimum-norm Solution Of Convex Qpsmentioning

confidence: 99%

On the minimum-norm solution of convex quadratic programming

2021

View full text Add to dashboard Cite

We discuss some basic concepts and present a numerical procedure for finding the minimum-norm solution of convex quadratic programs (QPs) subject to linear equality and inequality constraints. Our approach is based on a theorem of alternatives and on a convenient characterization of the solution set of convex QPs. We show that this problem can be reduced to a simple constrained minimization problem with a once-differentiable convex objective function. We use finite termination of an appropriate Newton's method to solve this problem. Numerical results show that the proposed method is efficient.

show abstract

“…The main issue in the convergence theory is to find conditions which are sufficient to verify the lower bounds in Theorem 1. However, it is well-known that due to the so-called Bakushinskii veto [1,17], a heuristic parameter choice functional cannot be a valid estimator for the stability error in the sense that (21) holds unless the permissible noise y δ − y is restricted in some sense. Conditions imposing such noise restrictions are at the heart of the convergence theory.…”

Section: Lower Bounds For ψ Slmentioning

confidence: 99%

“…with a general convex regularization functional R. For an analysis of several heuristic rules in this context, see [21]. Note that the L-curve method is then defined by analogy as a plot of (log(R(x δ α )), log( Ax δ α −y δ ).…”

Section: The Last Expressionmentioning

confidence: 99%

See 1 more Smart Citation

A simplified L-curve method as error estimator

Kindermann¹,

Raik²

2020

etna

Self Cite

View full text Add to dashboard Cite

The L-curve method is a well-known heuristic method for choosing the regularization parameter for ill-posed problems by selecting it according to the maximal curvature of the L-curve. In this article, we propose a simplified version that replaces the curvature essentially by the derivative of the parameterization on the y-axis. This method shows a similar behaviour to the original L-curve method, but unlike the latter, it may serve as an error estimator under typical conditions. Thus, we can accordingly prove convergence for the simplified L-curve method. is called the noise-level δ. In the case of heuristic parameter choice rules, which the L-curve method is an example of, this noise-level is considered unavailable.As the inverse of A is not bounded, the problem (1) cannot be solved by classical inversion algorithms, rather, a regularization scheme has to be applied [8]. That is, one constructs a one-parametric family of continuous operators (R α ) α , with α > 0, that in some sense approximates the inverse of A for α → 0.An approximation to the true solution of (1), denoted as x δ α , is computed by means of the regularization operators:A delicate issue in regularization schemes is the choice of the regularization parameter α and the standard methods make use of the noise-level δ. However, in situations when this is not available, so-called heuristic parameter choice methods [17] are proposed. The L-curve method selects an α corresponding to the corner point of the graph (log( Ax δ α − y δ ), log( x δ α )) parameterized by α. Recently, [17,19] a convergence theory for certain heuristic parameter choice rules was developed. Essential in this analysis is a restriction on the noise that rules out noise which is "too regular". Such noise conditions in the form of Muckenhoupt-type conditions were used in [17,19] and are currently the standard tool in the analysis of heuristic rules. If these conditions hold, then several well-known heuristic parameter choice rules serve as error estimators for the total error in typical regularization schemes and convergence and convergence rate results follow.The L-curve method, however, does not seem to be accessible to such an analysis, although some of its properties were investigated, for instance, by Hansen [13,15] and Reginska [29]. Nevertheless, it does not appear that it can be related to some sort of error estimators directly.There are various suggestions for efficient practical implementations of the L-curve method, like Krylov-space methods [6,30] or model functions [24]. Note that the method is also implemented in Hansen's Regularization Tools [14]. A generalization of the L-curve method in form of the Q-curve method was recently suggested by Raus and Hämarik [28]. Other simplifications or variations are the V-curve [9] or the U-curve [22]. Some overview and comparisons of other heuristic and non-heuristic methods are given in [2,11,12] and the PhD. thesis of Palm [26].The aim of this article is to propose a simplified version of the L-curve method by dropping several t...

show abstract

Convergence of Heuristic Parameter Choice Rules for Convex Tikhonov Regularization

Cited by 18 publications

References 24 publications

Estimating solution smoothness and data noise with Tikhonov regularization

Estimating solution smoothness and data noise with Tikhonov regularization

On the minimum-norm solution of convex quadratic programming

A simplified L-curve method as error estimator

Contact Info

Product

Resources

About