Predictive RANS simulations via Bayesian Model-Scenario Averaging

Edeling, Wouter; Cinnella, Paola; Dwight, Richard P.

doi:10.1016/j.jcp.2014.06.052

Cited by 104 publications

(81 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The majority of studies we encountred used an empirical approach to assess predictive performance, i.e., forecasting, hindcasting, or cross-validation to observed data (e.g., Namata et al 2008, Marmion et al 2009a,b, Grenouillet et al 2010, Montgomery et al 2012, Engler et al 2013, Smith et al 2013, Edeling et al 2014, Trolle et al 2014. Model averaging typically yielded lower prediction errors than the individual contributing models.…”

Section: Model Averaging (Typically) Reduces Prediction Errorsmentioning

confidence: 99%

Model averaging in ecology: a review of Bayesian, information‐theoretic, and tactical approaches for predictive inference

Dormann

Calabrese

Guillera‐Arroita

et al. 2018

Ecological Monographs

289

205

View full text Add to dashboard Cite

In ecology, the true causal structure for a given problem is often not known, and several plausible models and thus model predictions exist. It has been claimed that using weighted averages of these models can reduce prediction error, as well as better reflect model selection uncertainty. These claims, however, are often demonstrated by isolated examples. Analysts must better understand under which conditions model averaging can improve predictions and their uncertainty estimates. Moreover, a large range of different model averaging methods exists, raising the question of how they differ in their behaviour and performance. Here, we review the mathematical foundations of model averaging along with the diversity of approaches available. We explain that the error in model‐averaged predictions depends on each model's predictive bias and variance, as well as the covariance in predictions between models, and uncertainty about model weights. We show that model averaging is particularly useful if the predictive error of contributing model predictions is dominated by variance, and if the covariance between models is low. For noisy data, which predominate in ecology, these conditions will often be met. Many different methods to derive averaging weights exist, from Bayesian over information‐theoretical to cross‐validation optimized and resampling approaches. A general recommendation is difficult, because the performance of methods is often context dependent. Importantly, estimating weights creates some additional uncertainty. As a result, estimated model weights may not always outperform arbitrary fixed weights, such as equal weights for all models. When averaging a set of models with many inadequate models, however, estimating model weights will typically be superior to equal weights. We also investigate the quality of the confidence intervals calculated for model‐averaged predictions, showing that they differ greatly in behaviour and seldom manage to achieve nominal coverage. Our overall recommendations stress the importance of non‐parametric methods such as cross‐validation for a reliable uncertainty quantification of model‐averaged predictions.

show abstract

Section: Model Averaging (Typically) Reduces Prediction Errorsmentioning

confidence: 99%

Model averaging in ecology: a review of Bayesian, information‐theoretic, and tactical approaches for predictive inference

Dormann

Calabrese

Guillera‐Arroita

et al. 2018

Ecological Monographs

289

205

View full text Add to dashboard Cite

show abstract

“…A few recent studies have used Bayesian estimation techniques to calibrate RANS models. For example, Oliver and Moser [30], and Cheung et al [7] use Bayesian uncertainty analysis to calibrate and inter-compare four well-known RANS models: the Baldwin-Lomax model [3], the SA model [39], the Chien k-model [8], and the v 2 − f model [14]; Edeling et al [16,15] used Bayesian estimates of parameter variability in the Launder-Sharma k− model [23] as a means to estimate errors in RANS simulations; [17] present an adaptive modeling algorithm for selection and validation of models, however, in the domain of atomistic systems. In other work that uses a Bayesian framework in the context of RANS models, Edeling et al [16,15], note that the distribution of model parameters provides information on error associated with the model: when the joint probability density function (PDF) of the model parameters is propagated through the model, the distribution of the QoIs can be used to provide confidence bounds for the QoIs.…”

Section: Model Calibration and Validationmentioning

confidence: 99%

Leveraging Bayesian analysis to improve accuracy of approximate models

Nadiga

Jiang

Livescu

2019

Journal of Computational Physics

View full text Add to dashboard Cite

We focus on improving the accuracy of an approximate model of a multiscale dynamical system that uses a set of parameter-dependent terms to account for the effects of unresolved or neglected dynamics on resolved scales. We start by considering various methods of calibrating and analyzing such a model given a few well-resolved simulations. After presenting results for various point estimates and discussing some of their shortcomings, we demonstrate (a) the potential of hierarchical Bayesian analysis to uncover previously unanticipated physical dependencies in the approximate model, and (b) how such insights can then be used to improve the model. In effect parametric dependencies found from the Bayesian analysis are used to improve structural aspects of the model. While we choose to illustrate the procedure in the context of a closure model for buoyancy-driven, variable-density turbulence, the statistical nature of the approach makes it more generally applicable. Towards addressing issues of increased computational cost associated with the procedure, we demonstrate the use of a neural network based surrogate in accelerating the posterior sampling process and point to recent developments in variational inference as an alternative methodology for greatly mitigating such costs. We conclude by suggesting that modern validation and uncertainty quantification techniques such as the ones we consider have a valuable role to play in the development and improvement of approximate models.

show abstract

“…The few published studies do however illustrate the wide range of potential applications. Bayesian calibration was for instance used to assess the impact of the model constants of the k-ε turbulence model on the flow in a street canyon and on flat-plate boundary-layers (Edeling et al, 2014). Polynomial Chaos Expansions were used to model uncertainties in CFD calculations (Knio and Le Maître, 2006;Xiu and Karniadakis, 2003).…”

Section: Introductionmentioning

confidence: 99%