A Parsimonious Tour of Bayesian Model Uncertainty

Abstract

Modern statistical software and machine learning libraries are enabling semi-automated statistical inference. Within this context, it appears easier and easier to try and fit many models to the data at hand, reversing thereby the Fisherian way of conducting science by collecting data after the scientific hypothesis (and hence the model) has been determined. The renewed goal of the statistician becomes to help the practitioner choose within such large and heterogeneous families of models, a task known as model selection. The Bayesian paradigm offers a systematized way of assessing this problem. This approach, launched by Harold Jeffreys in his 1935 book Theory of Probability, has witnessed a remarkable evolution in the last decades, that has brought about several new theoretical and methodological advances. Some of these recent developments are the focus of this survey, which tries to present a unifying perspective on work carried out by different communities. In particular, we focus on non-asymptotic out-of-sample performance of Bayesian model selection and averaging techniques, and draw connections with penalized maximum likelihood. We also describe recent extensions to wider classes of probabilistic frameworks including high-dimensional, unidentifiable, or likelihood-free models.

Bayesian inference of multimessenger astrophysical data: Methods and applications to gravitational waves

“…If B B A < 1 then the hypothesis A is preferred by the data, otherwise B is favored if B B A > 1. However, this rule is not always straightforward since the estimation of the Bayes' factor might suffer of uncertainties [51,52]. Then, in a realistic scenario, more stringent bounds are required in order to prefer a hypothesis [53].…”

The Unreasonable Effectiveness of Deep Evidential Regression

“…One approach by Graves, Mohamed, and Hinton (2013) is to use a Gaussian variational posterior to approximate the distribution of the weights in a network, but the capacity of the uncertainty representation is limited by the variational distribution. In general we see that MCMC has a higher variance and lower bias in the estimate, while VI has a higher bias but lower variance (Mattei 2020). The preeminent Bayesian deep learning approach by Gal and Ghahramani (2016) showed that variational inference can be approximated without modifying the network.…”

${\tt bajes}$: Bayesian inference of multimessenger astrophysical data, methods and application to gravitational-waves

“…If B B A < 1 then the hypothesis A is preferred by the data, otherwise B is favored if B B A > 1. However, this rule is not always straightforward since the estimation of the Bayes' factor might suffer of uncertainties [51,52]. Then, more stringent bounds are required in order to prefer a hypothesis [53].…”