Partially Exchangeable Networks and Architectures for Learning Summary Statistics in Approximate Bayesian Computation

Abstract

We present a novel family of deep neural architectures, named partially exchangeable networks (PENs) that leverage probabilistic symmetries. By design, PENs are invariant to block-switch transformations, which characterize the partial exchangeability properties of conditionally Markovian processes. Moreover, we show that any block-switch invariant function has a PEN-like representation. The DeepSets architecture is a special case of PEN and we can therefore also target fully exchangeable data. We employ PENs to learn summary statistics in approximate Bayesian computation (ABC). When comparing PENs to previous deep learning methods for learning summary statistics, our results are highly competitive, both considering time series and static models. Indeed, PENs provide more reliable posterior samples even when using less training data.

Score Matched Neural Exponential Families for Likelihood-Free Inference

“…For the time series models, we used Partially Exchangeable Networks (PENs) [Wiqvist et al, 2019] to parametrize f w and s β . The key property of a r-PEN is that the transformation applied by the Neural Network is invariant to permutation of the input data which does not change the probability density of that data point, in the case in which the probability density is Markovian of order r (see Appendix D.3).…”

“…As AR(2) is a 2-Markovian model, we used a 2-PEN for f w . MA(2) is instead not a Markovian model; however, Wiqvist et al [2019] argued that it can be considered as "almost" Markovian, so that the loss of information in imposing a PEN structure of high enough order is negligible. Following them, we therefore used a 10-PEN.…”

“…In practice, Fearnhead and Prangle [2012] used a linear regression approach to solve the above, by considering s β (x) to be a linear function of the weights β. Jiang et al [2017], Wiqvist et al [2019], Åkesson et al [2020] instead use Neural Networks to parametrize s β and train them to minimize the loss in Eq. (1).…”

“…In order to reduce the dimensionality of the data, ABC usually relies on a set of summary statistics, whose choice is not straightforward. Recently, the expressive capabilities of Neural Networks (NNs) have been leveraged to learn statistics to be used in ABC [Jiang et al, 2017, Wiqvist et al, 2019, Åkesson et al, 2020. These techniques entail generating a training set of parameter-simulation pairs from the simulator, on which NNs parametrizing the statistics are trained by minimizing a suitable loss.…”

Cosmological Parameter Estimation and Inference using Deep Summaries

“…There are also combinations of ABC and deep neural networks, for example learning summary statistics for ABC inference has been proposed in [17][18][19] and a method to generate globally sufficient summaries of arbitrary data has been developed in [20]. However, these approaches usually require on-the-fly simulations and do not use the generated summaries for parameter estimation.…”