Robust Generalised Bayesian Inference for Intractable Likelihoods

Matsubara, Takuo; Knoblauch, Jeremias; Briol, François‐Xavier; Oates, Chris J.

doi:10.1111/rssb.12500

Cited by 20 publications

(21 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proof of Theorem 1 is provided in Appendix A.2. The result was established using similar arguments from early work by Hooker and Vidyashankar (2014); Ghosh and Basu (2016) and extended techniques of Miller (2021); Matsubara et al (2022). See also the recent review of Bochkina (2022).…”

Section: A Generalised Posteriorsupporting

confidence: 52%

“…To circumvent both computation of the normalising constant and simulation from the statistical model, Matsubara et al (2022) proposed a generalised Bayesian posterior, called KSD-Bayes, which is based on a Stein discrepancy. The resulting generalised posterior is consistent and asymptotically normal, and thus shares many of the properties of the standard Bayesian posterior whilst admitting a form which does not require the computation of an intractable normalisation constant.…”

Section: Introductionmentioning

confidence: 99%

“…Generalised Bayesian Inference Motivated by the absence of generally applicable computational methodology for intractable likelihood, Matsubara et al (2022) proposed a solution called KSD-Bayes. The setting for this approach is the nascent field of generalised Bayesian inference.…”

Section: Introductionmentioning

confidence: 99%

“…The standard Bayesian posterior is recovered by the negative log-likelihood D n (θ) = − n i=1 log p θ (x i ), while several alternative loss functions have been developed to confer robustness in settings where the statistical model is misspecified (see the survey in Bissiri et al (2016) for the case of additive loss functions, Jewson et al (2018) for the case for divergence-based loss functions, and for further generalisation). KSD-Bayes (Matsubara et al, 2022) is distinguished among existing generalised Bayesian inference methods by its applicability to statistical models involving an intractable normalising constant (see also Section 4.2 of Giummolè et al, 2019). This was achieved by selecting D n (θ) to be a Stein discrepancy between the statistical model p θ and the empirical distribution of the dataset, which can be computed without the normalising constant.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Generalised Bayesian Inference for Discrete Intractable Likelihood

Matsubara¹,

Knoblauch²,

Briol³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Discrete state spaces represent a major computational challenge to statistical inference, since the computation of normalisation constants requires summation over large or possibly infinite sets, which can be impractical. This paper addresses this computational challenge through the development of a novel generalised Bayesian inference procedure suitable for discrete intractable likelihood. Inspired by recent methodological advances for continuous data, the main idea is to update beliefs about model parameters using a discrete Fisher divergence, in lieu of the problematic intractable likelihood. The result is a generalised posterior that can be sampled using standard computational tools, such as Markov chain Monte Carlo, circumventing the intractable normalising constant. The statistical properties of the generalised posterior are analysed, with sufficient conditions for posterior consistency and asymptotic normality established. In addition, a novel and general approach to calibration of generalised posteriors is proposed. Applications are presented on lattice models for discrete spatial data and on multivariate models for count data, where in each case the methodology facilitates generalised Bayesian inference at low computational cost.theoretical guarantees, and can be computed in a cost O(nd) that is linear in the size of the dataset. Full details of the DFD-Bayes approach are provided next. MethodologyThis section presents and analyses DFD-Bayes. First, we present a novel discrete formulation of the Fisher divergence in Section 3.1. DFD-Bayes is introduced in Section 3.2, where posterior consistency and a Bernstein-von Mises result are established. Section 3.3 presents a novel approach to calibration of generalised posteriors, which may be of independent interest. Limitations of DFD-Bayes are discussed in Section 3.4.Notation Denote by X a countable set in which data are contained, and by Θ the set of permitted values for the parameter θ, where Θ is a Borel subset of R p for some p ∈ N. Probability distributions on X are identified with their probability mass functions, with respect to the counting measure on X . The i-th coordinate of a function f : X → R m is denoted by f i : X → R. For a probability distribution q on X and m, p ∈ N, denote by L p (q, R m ) the vector space of measurable functions f : X → R m such that m i=1 E X∼q [f i (X) p ] < ∞. For z ∈ R m , the Euclidean norm is denoted z . A Dirac measure at x ∈ X is denoted by δ x .

show abstract

Section: A Generalised Posteriorsupporting

confidence: 52%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Generalised Bayesian Inference for Discrete Intractable Likelihood

Matsubara¹,

Knoblauch²,

Briol³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…iv) This study has focused on estimation and computational methods proposed in the generalised Bayesian framework, in particular using robust divergence. On the other hand, methods using the Maximum Mean Discrepancy (Chérief-Abdellatif and Alquier, 2020) and Kernel Stein Discrepancy (Matsubara et al, 2022) have also been proposed in recent years in the same generalised Bayesian framework, although not with the motivation of dealing with outliers. Both require adjustment of the hyperparameters of the kernel used, and it may be possible to estimate them using our proposed method and compare their performance.…”

Section: Hertzsprung-russell Star Cluster Datamentioning

confidence: 99%

Adaptation of the Tuning Parameter in General Bayesian Inference with Robust Divergence

Yonekura

2022

Preprint

View full text Add to dashboard Cite

We introduce a novel methodology for robust Bayesian estimation with robust divergence (e.g., density power divergence or γ-divergence), indexed by tuning parameters. It is well known that the posterior density induced by robust divergence gives highly robust estimators against outliers if the tuning parameter is appropriately and carefully chosen. In a Bayesian framework, one way to find the optimal tuning parameter would be using evidence (marginal likelihood). However, we theoretically and numerically illustrate that evidence induced by the density power divergence does not work to select the optimal tuning parameter since robust divergence is not regarded as a statistical model. To overcome the problems, we treat the exponential of robust divergence as an unnormalisable statistical model, and we estimate the tuning parameter via minimizing the Hyvarinen score. We also provide adaptive computational methods based on sequential Monte Carlo (SMC) samplers, which enables us to obtain the optimal tuning parameter and samples from posterior distributions simultaneously. The empirical performance of the proposed method through simulations and an application to real data is also provided.

show abstract