An approach to quantifying the efficiency of a Bayesian filter

Abstract

[1] Data assimilation is the Bayesian conditioning of uncertain model simulations on observations to reduce uncertainty about model states. In practice, it is common to make simplifying assumptions about the prior and posterior state distributions, and to employ approximations of the likelihood function, which can reduce the efficiency of the filter. We propose metrics that quantify how much of the uncertainty in a Bayesian posterior state distribution is due to (i) the observation operator, (ii) observation error, and (iii) approximations of Bayes' Law. Our approach uses discrete Shannon entropy to quantify uncertainty, and we define the utility of an observation (for reducing uncertainty about a model state) as the ratio of the mutual information between the state and observation to the entropy of the state prior. These metrics make it possible to analyze the efficiency of a proposed observation system and data assimilation strategy, and provide a way to examine the propagation of information through the dynamic system model. We demonstrate the procedure on the problem of estimating profile soil moisture from observations at the surface (top 5 cm). The results show that when synthetic observations of 5 cm soil moisture are assimilated into a three-layer model of soil hydrology, the ensemble Kalman filter does not use all of the information available in observations.

Nonparametric triple collocation

“…That being said, it is our opinion that the philosophical argument is the most valuable contribution of this paper, as this piece of epistemology can be generalized to a wide array of statistical methods, models, and applications [e.g. , Nearing et al ., ; Nearing and Gupta , ].…”

“…We therefore imagine that most scientists would be interested in the empirical demonstrations because these show examples of how nonparametric TC is a practical methodology that is at least potentially more reliable than linear TC for at least certain applications. That being said, it is our opinion that the philosophical argument is the most valuable contribution of this paper, as this piece of epistemology can be generalized to a wide array of statistical methods, models, and applications [e.g., Nearing et al, 2013;Nearing and Gupta, 2015].…”

Reply to Pablo Durán-Barroso

“…However, strategies that result in model structural modifications can generally be expected to provide longer lasting benefits than ones that simply update or constrain the state trajectories of the model. This is because structural modifications can both improve the initial estimates of the state at each time step and sustain these improvements into future time steps (Bulygina and Gupta, 2009, 2010Nearing and Gupta, 2015). In contrast, even though data assimilation to directly adjust state estimates can improve model performance, inadequacies in model structure will tend to cause the state estimates to drift away from their more appropriate values over time, because of which the performance will deteriorate markedly when the constraining data are not available.…”

“…Alternatively, ET can be used as a calibration target along with streamflow within a multi-objective setting (Zhang et al, 2009). There has also been a recent drive towards structurally flexible models that are able to both better characterize the uncertainty associated with model structure and use additional information to help reduce such uncertainty (Wagener et al, 2001;Marshall et al, 2006;Clark et al, 2008Clark et al, , 2015Savenije, 2010;Schaefli et al, 2011;Fenicia et al, 2008aFenicia et al, , b, 2011Bulygina and Gupta, 2009, 2010Martinez and Gupta, 2011;Nearing, 2013;Nearing and Gupta, 2015).…”

“…The spatially lumped HYMOD Version 1 (HYMOD1) conceptual rainfall-runoff model with five/six parameters has previously been used in several studies (Boyle et al, 2000;Vrugt et al, 2003Vrugt et al, , 2009Moradkhani et al, 2005;Duan et al, 2007;Wang et al, 2009;Razavi and Gupta, 2016). The model is driven using daily precipitation and PET data to generate daily estimates of AET (HAET: HYMODgenerated AET) and streamflow.…”