Why Curvature in L-Curve: Combining Soft Constraints

Abstract

Abstract.In solving inverse problems, one of the successful methods of determining the appropriate value of the regularization parameter is the L-curve method of combining the corresponding soft constraints, when we plot the curve describing the dependence of the logarithm x of the mean square difference on the logarithm y of the mean square non-smoothness, and select a point on this curve at which the curvature is the largest. This method is empirically successful, but from the theoretical viewpoint, it is not clear why we should use curvature and not some other criterion. In this paper, we show that reasonable scale-invariance requirements lead to curvature and its generalizations.

Constraint Programming and Decision Making

“…Instead of combining criteria and solving the resulting combined optimization problem, we can alternatively solve the optimization problems corresponding to all possible combinations, and then select the solution which is, in some reasonable sense, the most appropriate. As shown in [21], in this case also natural symmetries explain the efficiency of empirically successful selection heuristics.…”

A Bayesian interpretation of the L-curve

“…A more formal justification is as follows. Let us introduce the (squared) Euclidian distance, 2 , between two points P 1 (ζ 1 ; η 1 ) and P 2 (ζ 2 ; η 2 ) in plane P. Substituting expressions (27) for ζ and η, this is also the (squared) Riemannian distance, ln 2 [38]-a scale invariant distance that depends only on the shape of Φ N and Φ X . Now let T t = T(X(λ(t))) and U t = U(X(λ(t))) for notational simplicity.…”

“…One rationale beyond the maximum curvature criterion is its invariance with respect to re-scaling of the regularization parameter λ. Interestingly, Ref. [2] proves that a criterion for selecting the regularization parameter is invariant under i) scale transformation of the data and ii) re-scaling of the regularization parameter if and only if it is a function of κ and ζ /η only, with f = ln. However, the scale invariance of λ (ii) is a questionable property that will be reappraised in section 3 of this paper.…”

“…Quantiles of estimates of the regularization parameter: λ SE stands the reference that minimizes the square error X − X(λ)2 F , λ BL κ for the maximum curvature estimate of the Bayesian L-curve, λ BL J1 for the minimum of J 1 , and λ κ for the maximum curvature estimate of the reference L-curve. Boxplots (marking the lower, middle, and upper quartiles and whiskers set to four times the interquartile range) of the NSE (estimated on 10 4 runs) for increasing values of the ratio M/K: λ MSE , λ BL κ , λ BL J1 , and λκ are as defined in table 1.…”

“…quantile log 10 λ SE log 10 λ BL Quantiles of estimates of the regularization parameter: λ SE stands the reference that minimizes the square error X−X(λ)2 F , λ BL κ for the maximum curvature estimate of the Bayesian L-curve, λ BL J 1 for the minimum of J 1 , and λ κ for the maximum curvature estimate of the reference L-curve.…”