Discontinuous parameter estimates with least squares estimators

Abstract

We discuss weighted least squares estimates of ill-conditioned linear inverse problems where weights are chosen to be inverse error covariance matrices. Least squares estimators are the maximum likelihood estimate for normally distributed data and parameters, but here we do not assume particular probability distributions. Weights for the estimator are found by ensuring its minimum follows a χ 2 distribution. Previous work with this approach has shown that the it is competitive with regularization methods such as the L-curve and Generalized Cross Validation (GCV) [23]. In this work we extend the method to find diagonal weighting matrices, rather than a scalar regularization parameter. Diagonal weighting matrices are advantageous because they give piecewise smooth least squares estimates and hence are a mechanism through which least squares can be used to estimate discontinuous parameters. This is explained by viewing least squares estimation as a constrained optimization problem. Results with diagonal weighting matrices are given for a benchmark discontinuous inverse problem from [15]. In addition, the method is used to estimate soil moisture from data collected in the Dry Creek Watershed near Boise, Idaho. Parameter estimates are found that combine two different types of measurements, and weighting matrices are found that incorporate uncertainty due to spatial variation so that the parameters can be used over larger scales than those that were measured.

$\chi^2$ Tests for the Choice of the Regularization Parameter in Nonlinear Inverse Problems

“…Future work includes estimating more complex covariance matrices for the parameter estimates. In [9] Mead shows that it is possible to use multiple χ 2 tests to estimate such a covariance, and it seems likely that this could also be extended to solving nonlinear problems.…”

“…It is shown there that this is an attractive alternative to the Lcurve, GCV, and the discrepancy principle among other methods. Preliminary work for more dense estimates of C or C f has been done in [7,9], and future work involves efficient solution of nonlinear systems similar to (2.3) to estimate more dense C f or C .…”

Regularization parameter estimation for underdetermined problems by the χ 2 principle with application to 2D focusing gravity inversion

“…The UPRE, GCV and χ 2 -principle algorithms for estimating a regularization parameter in the context of underdetermined Tikhonov regularization have been developed and investigated, extending the χ 2 method discussed in [13,14,15,16,17]. UPRE and χ 2 techniques require that an estimate of the noise distribution in the data measurements is available, while ideally the χ 2 also requires a prior estimate of the mean of the solution in order to apply the central version of the χ 2 algorithm.…”

“…(16) A root finding algorithm for c = 0 and W L = σ −2 L I was presented in [16], and extended for c > 0 in [22]. The general and difficult multi-parameter case was discussed in [14], with extensions for nonlinear problems in [15]. We collect all the parameter estimation formulae in Appendix A.…”

“…More recently, a new approach based on the χ 2 property of the functional (3) under statistical assumptions applied through C L , was proposed by [13], for the overdetermined case m > n with p = n. The extension to the case with p ≤ n and a discussion of effective numerical algorithms is given in [16,22], with also consideration of the case whenm = m 0 is not available. Extensions for nonlinear problems [15], inclusion of inequality constraints [17] and for multi parameter assumptions [14] have also been considered.…”