On the safe use of prior densities for Bayesian model selection

Abstract

The application of Bayesian inference for the purpose of model selection is very popular nowadays. In this framework, models are compared through their marginal likelihoods, or their quotients, called Bayes factors. However, marginal likelihoods depends on the prior choice. For model selection, even diffuse priors can be actually very informative, unlike for the parameter estimation problem. Furthermore, when the prior is improper, the marginal likelihood of the corresponding model is undetermined. In this work, we discuss the issue of prior sensitivity of the marginal likelihood and its role in model selection. We also comment on the use of uninformative priors, which are very common choices in practice. Several practical suggestions are discussed and many possible solutions, proposed in the literature, to design objective priors for model selection are described. Some of them also allow the use of improper priors. The connection between the marginal likelihood approach and the well-known information criteria is also presented. We describe the main issues and possible solutions by illustrative numerical examples, providing also some related code. One of them involving a real-world application on exoplanet detection.

Proximal nested sampling for high-dimensional Bayesian model selection

“…Important perspectives for future work include: a detailed theoretical analysis of the convergence properties of proximal nested sampling; an extension to (biased) accelerated proximal methods (Vargas et al 2020); and an analysis of the properties of marginal maximum likelihood estimation for the class of models considered in this paper, such as estimator consistency for model selection in an M-closed setting and concentration in an M-open setting (Llorente et al 2022). Moreover, it would be interesting to apply proximal nested sampling to other types of models, such as models with likelihood-based priors (Llorente et al 2022), which can be handled straightforwardly by proximal nested sampling when the likelihood is log-concave. It would also be interesting to modify proximal nested sampling to tackle high-dimensional models that are multi-modal, particularly models with data-driven priors encoded by neural networks (see e.g.…”

“…Moreover, since the proposed proximal nested sampling approach was specifically designed for models that are logconcave and with Bayesian imaging applications in mind, we anticipate that it will be mostly used with informative priors designed to regularise and stabilise high-dimensional estimation problems. As explained in Llorente et al (2022), the marginal likelihood can be very sensitive to the choice of the prior. Therefore, it is important that the parameters of the prior are chosen carefully.…”

“…Consequently, when using proximal nested sampling in this context one is inherently operating in an M-open Bayesian modelling paradigm, where none of the models under consideration are formally "true". We refer the reader to Llorente et al (2022) for more details about performing model selection in this context, as well as for details about prior sensitivity, objectivity, and the use of data-driven priors in Bayesian model selection.…”

Gaussian Processes and Nested Sampling Applied to Kepler's Small Long-period Exoplanet Candidates

“…Varying searchable parameter space does not pose much of an issue apart from sampling inefficiencies when employing informative priors, as exploration beyond their regions of significant probability density with naturally welldefined support returns little to no information. The same cannot be said for uniform priors, whose normalization biases measurements of the  via Equation (2) (for a detailed study regarding priors and caveats such as this within the context of Bayesian inference and model selection, see Llorente et al 2022).…”