Considerate approaches to constructing summary statistics for ABC model selection

Abstract

For nearly any challenging scientific problem evaluation of the likelihood is problematic if not impossible. Approximate Bayesian computation (ABC) allows us to employ the whole Bayesian formalism to problems where we can use simulations from a model, but cannot evaluate the likelihood directly. When summary statistics of real and simulated data are compared -rather than the data directly -information is lost, unless the summary statistics are sufficient. Here we employ an information-theoretical framework that can be used to construct (approximately) sufficient statistics by combining different statistics until the loss of information is minimized. Such sufficient sets of statistics are constructed for both parameter estimation and model selection problems. We apply our approach to a range of illustrative and real-world model selection problems.

On selection of statistics for approximate Bayesian computing (or the method of simulated moments)

“…As an example of papers that select a subset of statistics, Joyce and Marjoram (2008) propose an algorithm to determine whether or not a new statistic should be added to a previously selected set of statistics, based on approximations of the posterior odds ratio, In particular, our procedure does not involve any additional tuning parameters (besides, optionally, the penalization weight). Below, we discuss how the information theoretic criterion developed in Barnes, et al (2012) in principle could be combined with the ideas of this paper to yield an alternative version of our cross-validation method.…”

“…Thus, compared to the selection criterion proposed in the previous section, this KLIC selection rule satisfies the stronger property of Z 0 being a sufficient statistic relative to the candidate set of statistics W. The use of KLIC for selecting summary statistics was already proposed by Barnes et al (2012); however, while they use a step-up procedure for identifying δ 0 , where one statistic is added at a time until there is no further improvement in terms of KLIC, we here do a full search over all possible sets of candidates.…”

“…As already noted by Barnes et al (2012), KLIC is only of theoretical interest since its computation involves approximating f (θ|w) which is a high-dimensional problem when dim W is large. To resolve this issue, we proceed as in the previous section and introduce the integrated (or expected) version of this measure,…”

“…z. This version of our selection procedure chooses the set of summary statistics that maximizes the information content over the whole distribution and so is similar in spirit to the one of Barnes et al (2012). However, the procedure of Barnes et al (2012) only performs a partial search over the set of candidate statistics, while our algorithm does a full search.…”

Approximate Bayesian Computation and Simulation-Based Inference for Complex Stochastic Epidemic Models

“…Increasing research effort has been placed into deriving approximately sufficient summary measures [e.g. Joyce and Marjoram, 2008, Nunes and Balding, 2010, Barnes et al, 2012, Fearnhead and Prangle, 2012, Ratmann et al, 2014. The use of summary statistics can also be extended to indirect inference methods, where the auxiliary models describe the distributions of the summary statistics (see Section 2.7).…”