A rare event approach to high-dimensional approximate Bayesian computation

Abstract

Approximate Bayesian computation (ABC) methods permit approximate inference for intractable likelihoods when it is possible to simulate from the model. However, they perform poorly for high-dimensional data and in practice must usually be used in conjunction with dimension reduction methods, resulting in a loss of accuracy which is hard to quantify or control. We propose a new ABC method for high-dimensional data based on rare event methods which we refer to as RE-ABC. This uses a latent variable representation of the model. For a given parameter value, we estimate the probability of the rare event that the latent variables correspond to data roughly consistent with the observations. This is performed using sequential Monte Carlo and slice sampling to systematically search the space of latent variables. In contrast, standard ABC can be viewed as using a more naive Monte Carlo estimate. We use our rare event probability estimator as a likelihood estimate within the pseudo-marginal Metropolis-Hastings algorithm for parameter inference. We provide asymptotics showing that RE-ABC has a lower computational cost for high-dimensional data than standard ABC methods. We also illustrate our approach empirically, on a Gaussian distribution and an application in infectious disease modelling.Electronic supplementary material The online version of this article

Approximate Bayesian Computation with the Wasserstein Distance

“…Suppose that is chosen asThen, the resulting ABC posterior can be shown to have the desirable theoretical property of converging to the standard posterior as ɛ →0 (Prangle et al . (); see also proposition 2 in Section 3.1). In the case where p =2, and , is a scaled version of the Euclidean distance between the vectors and z 1: n .…”

“…In some cases, observations from the model are obtained as z 1:n = g n .u, θ/, where g n is a known deterministic function and u some known fixed dimensional random variable independent of θ. Some methods require access to g n and u (Prangle et al, 2016;Graham and Storkey, 2017); by contrast, here we do not place assumptions on how data sets are generated from the model. studying ABC posteriors in the setting where D is the Euclidean distance between summaries, i.e.…”

Delayed Acceptance ABC-SMC

“…The first example of this simplification is that when used in the ABC-SMC method of Del Moral et al (2012a), the particle weights are either zero ("dead") or non-zero ("alive") (this following from equation ( 7)). This type of SMC algorithm is also studied in Cérou et al (2012);Del Moral et al (2015); Prangle et al (2017). A further consequence that the ESS is equal to the number of "alive" particles with non-zero weight after the update, simplifying the interpretation of the adaptive approach to choosing t+1 described above.…”

“…Therefore Bernton et al (2017) suggest instead to choose t+1 such that some number U (with 0 < U ≤ N ) of the N particles will be unique after resampling has been performed. Both this new scheme, and the original method that uses the ESS, introduce a small bias into estimates from the SMC (Cérou et al, 2012;Prangle et al, 2017).…”

Weighted approximate Bayesian computation via Sanov’s theorem

“…In order to tackle the problem more systematically, clever proposal distributions should be combined with better approximations to the likelihood. Accordingly, Prangle et al (2018) also resorted to a sequential approach, but explicitly considering a likelihood estimate that takes into account the probability of rare events. As a comparison, our method evaluates the probabilities of rare events based on theoretical results (LDT), rather than on Monte Carlo estimates of tail probabilities.…”