Minimum relative entropy and probabilistic inversion in groundwater hydrology

Abstract

The similarity between maximum entropy (MaxEnt) and minimum relative entropy (MRE) allows recent advances in probabilistic inversion to obviate some of the shortcomings in the former method. The purpose of this paper is to review and extend the theory and practice of minimum relative entropy. In this regard, we illustrate important philosophies on inversion and the similarly and differences between maximum entropy, minimum relative entropy, classical smallest model (SVD) and Bayesian solutions for inverse problems. MaxEnt is applicable when we are determining a function that can be regarded as a probability distribution. The approach can be extended to the case of the general linear problem and is interpreted as the model which ®ts all the constraints and is the one model which has the greatest multiplicity or`s preadout'' that can be realized in the greatest number of ways. The MRE solution to the inverse problem differs from the maximum entropy viewpoint as noted above. The relative entropy formulation provides the advantage of allowing for non-positive models, a prior bias in the estimated pdf and`hard' bounds if desired. We outline how MRE can be used as a measure of resolution in linear inversion and show that MRE provides us with a method to explore the limits of model space. The Bayesian methodology readily lends itself to the problem of updating prior probabilities based on uncertain ®eld measurements, and whose truth follows from the theorems of total and compound probabilities. In the Bayesian approach information is complete and Bayes' theorem gives a unique posterior pdf. In comparing the results of the classical, MaxEnt, MRE and Bayesian approaches we notice that the approaches produce different results. In comparing MaxEnt with MRE for Jayne's die problem we see excellent 317 comparisons between the results. We compare MaxEnt, smallest model and MRE approaches for the density distribution of an equivalent spherically-symmetric earth and for the contaminant plume-source problem. Theoretical comparisons between MRE and Bayesian solutions for the case of the linear model and Gaussian priors may show different results. The Bayesian expected-value solution approaches that of MRE and that of the smallest model as the prior distribution becomes uniform, but the Bayesian maximum aposteriori (MAP) solution may not exist for an underdetermined case with a uniform prior. Contents

A full‐Bayesian approach to the groundwater inverse problem for steady state flow

“…It is important to note that the errors produced are a reflection of the errors in the measurements translated through the appropriate nonlinear kernel that governs the problem along with the regularizer. These errors do not reflect the spread of reasonably probable models, which is one of the main goals in inverse problems for site characterization [Woodbury and Ulrych, 1998]. …”

“…This viewpoint is essentially Bayesian and is readily applicable to the questions that scientists and engineers typically ask. A necessary component of the Jaynes-Cox view is the "principle of maximum entropy" (PME) which replaces the need for subjective prior information in the Bayesian approach and forces all observers who possess common information to produce consistent results [Woodbury and Ulrych, 1998]. It is important to note that the above approach (PME) of determiningp (in) is the one which is the most uncommitted with respect to unknown information.…”

L-moments and C-moments

“…minimum and maximum values of the sample), and the prior is often the uniform distribution on aY b (the least informative prior). A particularly¯exible method of incorporating prior information is by means of the principle of minimum relative entropy, that has found important application in the inversion of linear problems of general interest (see Woodbury and Ulrych, 1998 for a complete review).…”

On the Use of Information Theory to Quantify Parameter Uncertainty in Groundwater Modeling

“…This method presumes knowledge of a prior probability distribution and produces the posterior probability distribution based on the information provided by new moments. Details of the minimum relative entropy are found in [16].…”

“…The use of maximum entropy in hydrology is somewhat limited to evaluate the efficiency of monitoring networks [14] such as a water quality network [15]. Woodbury and Ulrych [16] developed a different approach of entropy estimation called minimum relative entropy for groundwater models. In the minimum relative entropy, knowledge of moments is used as "data" rather than sample values.…”