Efficient learning methods for large-scale optimal inversion design

Abstract

In this work, we investigate various approaches that use learning from training data to solve inverse problems, following a bilevel learning approach. We consider a general framework for optimal inversion design, where training data can be used to learn optimal regularization parameters, data fidelity terms, and regularizers, thereby resulting in superior variational regularization methods. In particular, we describe methods to learn optimal p and q norms for L p − L q regularization and methods to learn optimal parameters for regularization matrices defined by covariance kernels. We exploit efficient algorithms based on Krylov projection methods for solving the regularized problems, both at training and validation stages, making these methods well-suited for large-scale problems. Our experiments show that the learned regularization methods perform well even when there is some inexactness in the forward operator, resulting in a mixture of model and measurement error.

The Variable Projected Augmented Lagrangian Method

“…We will extend the DF argument to be used to find µ using a standard bisection algorithm at relatively low cost, when the original model parameters can be assumed to be differentially Laplacian, as is the case for standard image deblurring problems. Fourth, we consider 2 − 1 norm regularization here; however, the developed approach extends also to other p − q norm problems [13] and even more general objective functions. We will investigate the convergence and numerical advantages and disadvantages of utilizing a variable projected approach for such p − q problems and for supervised learning loss function fitting within this framework.…”

The Image Deblurring Problem: Matrices, Wavelets, and Multilevel Methods

“…Other well-known methods for choosing the parameter λ include the Generalized Cross Validation (GCV) [10], that chooses λ to maximize the accuracy with which we can predict the value of a pixel that has been omitted, the unbiased predictive risk estimator (UPRE) [19], and more recently, methods based on learning when training data is available [4,5].…”