Maximum likelihood Bayesian averaging of airflow models in unsaturated fractured tuff using Occam and variance windows

Abstract

We use log permeability and porosity data obtained from single-hole pneumatic packer tests in six boreholes drilled into unsaturated fractured tuff near Superior, Arizona, to postulate, calibrate and compare five alternative variogram models (exponential, exponential with linear drift, power, truncated power based on exponential modes, and truncated power based on Gaussian modes) of these parameters based on four model selection criteria (AIC, AICc, BIC and KIC). Relying primarily on KIC and cross-validation we select the first three of these variogram models and use them to parameterize log air permeability and porosity across the site via kriging in terms of their values at selected pilot points and at some single-hole measurement locations. For each of the three variogram models we estimate log air permeabilities and porosities at the pilot points by calibrating a finite volume pressure simulator against two cross-hole pressure data sets from sixteen boreholes at the site. The traditional Occam's window approach in conjunction with AIC, AICc, BIC and KIC assigns a posterior probability of nearly 1 to the power model. A recently proposed variance window approach does the same when applied in conjunction with AIC, AICc, BIC but spreads the posterior probability more evenly among the three models when used in conjunction with KIC. We compare the abilities of individual models and MLBMA, based on both Occam and variance windows, to predict space-time pressure variations observed during two cross-hole tests other than those employed for calibration. Individual models with the largest posterior probabilities turned out to be the worst or second worst predictors of pressure in both validation cases. Some individual models predicted pressures more accurately than did MLBMA. MLBMA was far superior to any of the individual models in one validation test and second to last in the other validation test in terms of predictive coverage and log scores.

Finding the right balance between groundwater model complexity and experimental effort via Bayesian model selection

“…The Kashyap information criterion (KIC) (Neuman, 2003), the Bayesian information criterion (BIC) (Schwarz, 1978;Raftery, 1995), and the Akaike information criterion (AIC) (Akaike, 1973) are the most commonly used ones within the BMA framework, although the AIC does not originate from the Bayesian context (e.g., Burnham and Anderson, 2004). While these criteria are favorable due to their computational efficiency, they have been shown to yield inaccurate and contradicting model ranking results in numerous studies (see e.g., Ye et al, 2008;Tsai and Li, 2008;Ye et al, 2010;Singh et al, 2010;Morales-Casique et al, 2010;Foglia et al, 2013). In a benchmarking exercise (Schöniger et al, 2014), these information criteria have been tested against computationally expensive numerical reference solutions for BME in both synthetic and real-world cases.…”

“…In groundwater modeling, it has been applied to choose between different parameterizations of aquifer heterogeneity, e.g. by Ye et al (2004), Tsai and Li (2008), Rojas et al (2008), Morales-Casique et al (2010), Seifert et al (2012), and Elsheikh et al (2013), to name only a few selected examples. Refsgaard et al (2012) provide a review of strategies, including BMA, to address geological uncertainty in groundwater flow and transport modeling.…”

Effects of error covariance structure on estimation of model averaging weights and predictive performance

“…Similar situations occurred in Morales‐Casique et al . [] when studying a number of geostatistical and air flow models, in Diks and Vrugt [] for two cases that involved eight watershed models and seven soil hydraulic models, respectively, and in Seifert et al . [] for six hydrological models with different conceptual geological configurations.…”

Multi-Scale Assessment of Prediction Uncertainty in Coupled Reactive Transport Models Conducted at the Florida State University

“…The model accurately represents the system; [35] 2. Model prediction g(b) is monotonic enough that any local extreme of g(b) within the closed parameter regions lies between the maximum and minimum values of g(b) that occur along the boundary of the regions [Cooley, 1993a]; [36] 3. There is a single minimum in the objective function;…”

“…Postulate a set M of K mutually independent geostatistical, statistical or stochastic models, M k , with parameters h k for the desired output vector, D; 2. Obtain ML estimatesĥ [36] or via Monte Carlo simulation (used in this study): a. Draw random samples (realizations) of h k from a multivariate Gaussian distribution with meanĥ…”