Abstract.A standard approach to solving inversion problems that involve many parameters uses gradient-based optimization to find the parameters that best match the data. We will discuss enabling techniques that facilitate application of this approach to large-scale computational simulations, which are the only way to investigate many complex physical phenomena. Such simulations may not seem to lend themselves to calculation of the gradient with respect to numerous parameters. However, adjoint differentiation allows one to efficiently compute the gradient of an objective function with respect to all the variables of a simulation. When combined with advanced gradient-based optimization algorithms, adjoint differentiation permits one to solve very large problems of optimization or parameter estimation. These techniques will be illustrated through the simulation of the time-dependent diffusion of infrared light through tissue, which has been used to perform optical tomography. The techniques discussed have a wide range of applicability to modeling including the optimization of models to achieve a desired design goal.Key words: simulation, inversion, reconstruction, adjoint differentiation, sensitivity analysis, optimization, model validation
INTRODUCTION TO THE GENERAL PROBLEMFrequently a physical situation can only be described fully by a computational model. We wish to address the general problem of finding the values of the parameters in such a model that best match a given set of data. This problem in often referred to as that of inversion. In data matching the objective function to be minimized is often the negative logarithm of the likelihood of the data given their predicted values, which yields the maximum likelihood (ML) solution. Alternative approaches include regularized versions of maximum likelihood and Bayesian methods, in which the objective function is the minus-log-posterior, yielding the maximum a posteriori (MAP) estimate.We confine ourselves to objective functions that depend on the parameters in a continuous and differentiable fashion. We do not necessarily avoid problems for which the objective function possesses multiple minima. However, because the techniques that we present make use of gradients in the optimization process, they will work effectively only when one can easily find the basin of attraction for the global minimum, for example, by multiscale or multiresolution optimization.