A synthetic groundwater modelling study of the accuracy of GLUE uncertainty intervals

Abstract

Synthetic groundwater flow models with one unknown parameter, the average log transmissivity of the flow domain, and with Gaussian log-transmissivity error structure were used to study the nature and the accuracy of Generalized Likelihood Uncertainty Estimation (GLUE) intervals. The uniform prior distribution of log10 transmissivities was sampled uniformly and 1000 values per log10-transmissivity cycle were required to produce unbiased GLUE results. Because the errors in hydraulic head resulting from the log-transmissivity errors are known to be Gaussian for a linear model for heads, the Gaussian likelihood function was used as the GLUE goodness-of-fit function in most cases studied. The GLUE interval computed for the hydraulic head at different locations within the domain has the characteristics of a confidence interval for the hydraulic heads computed using the spatial average log transmissivity. The GLUE interval does not have the characteristics of a prediction interval, which is a probability interval for an uncertain observation of some variable such as the hydraulic head. The goodness-of-fit function can be corrected so that the resulting GLUE interval has the characteristics of a prediction interval. However, neither the original nor the corrected GLUE interval account for the uncertainty caused by small-scale model errors, which is always present in practical groundwater flow modelling. It is therefore concluded that one should be careful with using the GLUE interval to evaluate the validity of a model's structure. The structure may be valid even though observations fall significantly outside the GLUE interval if the observations are uncertain and the goodness-of-fit function is not corrected to account for this, or if small-scale model error is significant. If small-scale model error does not significantly bias model predictions, the predictions will be useful although uncertain. Small-scale model error did not bias the predictions in the examples studied here except near a strong sink. Changing the goodness-of-fit function from the Gaussian likelihood function to a similar but different function made the resulting GLUE intervals very inaccurate. Changing to a third function based on a fitted model rejection value produced results that were somewhat better than those obtained with the Gaussian likelihood function. However, the results were sensitive to the model rejection value, which in the ideal case should be adjusted for each predicted variable individually. Thus, the third function is not attractive for practical applications with the GLUE methodology.

Equifinality of formal (DREAM) and informal (GLUE) Bayesian approaches in hydrologic modeling?

“…Recent contributions to the hydrologic literature have criticized GLUE for not being formally Bayesian, resulting in parameter and predictive distributions that are statistically incoherent, unreliable, and that should therefore not be used (Christensen 2004;Montanari 2005;Mantovan and Todini 2006;Vogel et al 2008). The GLUE method is most often used with a statistically informal likelihood function, does not attempt to find the maximum likelihood estimate of the parameters to benchmark the performance of the best model, and does not explicitly consider model errors in the derivation and communication of predictive distributions.…”

Bridging the gap between GLUE and formal statistical approaches: approximate Bayesian computation

“…In recent years, a strong debate has emerged in the hydrologic community between proponents that adhere strongly to the underlying philosophy of GLUE and believe that the method is a useful working methodology for assessing parameter and predictive uncertainty in nonideal cases, and researchers and practitioners that strongly oppose incorrect usage of statistics in favor of coherent probabilistic approaches Beven and Young, 2003;Gupta et al, 2003;Christensen, 2004;Montanari, 2005;Mantovan and Todini, 2006;Stedinger et al, 2008;Beven et al, 2008;Beven, 2009;Vrugt et al, 2008b, c). This paper is a followup of our earlier work and demonstrates the similarity of likelihood-free inference used in population and evolutionary genetics (Pritchard et al, 1999;Beaumont et al, 2002) and informal statistical approaches such as GLUE.…”

Quantifying soil hydraulic properties and their uncertainties by modified GLUE method

“…Various likelihood measures have been put forward and evaluated in the literature (Christensen, 2004;Stedinger, 2008). In this paper the likelihood measure is the one most often applied, introduced by Nash and Sutcliffe (1970):…”

“…On the other hand, in recent years some authors criticised GLUE for not being formal Bayesian, and requiring subjective decisions on the likelihood measure and threshold (Christensen, 2004;Mantovan and Todini, 2006;Montanari, 2005). Blasone et al (2008) deemed that GLUE was inapplicable for high-dimensional and complex estimation, because it cannot achieve robust and consistent estimates of behavioural models.…”