Relevant Statistics for Bayesian Model Choice

Abstract

International audienceThe choice of the summary statistics that are used in Bayesian inference and in particular in approximate Bayesian computation algorithms has bearings on the validation of the resulting inference. Those statistics are nonetheless customarily used in approximate Bayesian computation algorithms without consistency checks. We derive necessary and sufficient conditions on summary statistics for the corresponding Bayes factor to be convergent, namely to select the true model asymptotically. Those conditions, which amount to the expectations of the summary statistics differing asymptotically under the two models, are quite natural and can be exploited in approximate Bayesian computation settings to infer whether or not a choice of summary statistics is appropriate, via a Monte Carlo validation

Approximate Bayesian Computation: A Survey on Recent Results

“…Even though the idea of separating the mean behaviour of the summary statistic under both models is intuitive, establishing a complete theoretical framework that validated this intuition requires assumptions borrowed from the asymptotic Bayesian literature [66]. The main theorem in [38] states that, under such assumptions, when the "true" mean E[S(Y )] of the summary statistic can be recovered for both models under comparison, then the Bayes factor is of order…”

“…The subsequent [38] deals with the contrasted performances and the resulting evaluation of summary statistics for Bayesian model choice (and not solely in ABC settings). The central result in this paper is that the summary statistic should enjoy a different range of means (as a vector) under different models for the corresponding Bayes factor to be consistent (as the number of observations goes to zero).…”

On some recent advances on high dimensional Bayesian statistics

“…Similarly, we will not address issues regarding how to enhance efficiency of abc and its variants, as for example with the sequential techniques of [SFT07] and [BCMR09]. Nor won't we explore the important question of abc model choice, for which theoretical arguments are still missing [RCMP11,MPRR11]. Finally, we refer the reader to [BCG12] for details and proofs concerning the upcoming results.…”

The expected demise of the Bayes factor

“…In a recent work about the validation of ABC model selection Marin et al, 2014), we also pointed out the variability of the numerical estimates and in fine the utter dependence of both posterior probabilities and Bayes factors on conditioning statistics, which in turn undermines their validity for model assessment.…”