Informed sub-sampling MCMC: approximate Bayesian inference for large datasets

Maire, Florian; Friel, Nial; Alquier, Pierre

doi:10.1007/s11222-018-9817-3

Cited by 17 publications

(13 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the total variation metric is too difficult to directly estimate in higher dimensions, we follow the standard approach in the literature (see for example Durmus and Moulines [2016] and Maire et al [2018]) and consider instead the mean marginal total variation (MMTV),…”

Section: Numerical Experimentsmentioning

confidence: 99%

Implicit Langevin Algorithms for Sampling From Log-concave Densities

Hodgkinson¹,

Salomone²,

Roosta³

2019

Preprint

View full text Add to dashboard Cite

For sampling from a log-concave density, we study implicit integrators resulting from θ-method discretization of the overdamped Langevin diffusion stochastic differential equation. Theoretical and algorithmic properties of the resulting sampling methods for θ ∈ [0, 1] and a range of step sizes are established. Our results generalize and extend prior works in several directions. In particular, for θ ≥ 1/2, we prove geometric ergodicity and stability of the resulting methods for all step sizes. We show that obtaining subsequent samples amounts to solving a strongly-convex optimization problem, which is readily achievable using one of numerous existing methods. Numerical examples supporting our theoretical analysis are also presented.

show abstract

Section: Numerical Experimentsmentioning

confidence: 99%

Implicit Langevin Algorithms for Sampling From Log-concave Densities

Hodgkinson¹,

Salomone²,

Roosta³

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Other authors use a subsample of the data in each MCMC iteration to speed up the algorithm, see e.g. Korattikara et al (2014), Bardenet et al (2014), Maclaurin and Adams (2014), Maire et al (2015), Bardenet et al (2015) and Quiroz et al (2016Quiroz et al ( , 2017. Finally, delayed acceptance MCMC has been used to speed up computations (Banterle et al, 2014;Payne and Mallick, 2015).…”

Section: Introductionmentioning

confidence: 99%

Speeding up MCMC by Delayed Acceptance and Data Subsampling

Quiroz

Tran

Villani

et al. 2017

Journal of Computational and Graphical Statistics

View full text Add to dashboard Cite

Abstract. The complexity of the Metropolis-Hastings (MH) algorithm arises from the requirement of a likelihood evaluation for the full data set in each iteration. Payne and Mallick (2015) propose to speed up the algorithm by a delayed acceptance approach where the acceptance decision proceeds in two stages. In the first stage, an estimate of the likelihood based on a random subsample determines if it is likely that the draw will be accepted and, if so, the second stage uses the full data likelihood to decide upon final acceptance. Evaluating the full data likelihood is thus avoided for draws that are unlikely to be accepted.We propose a more precise likelihood estimator which incorporates auxiliary information about the full data likelihood while only operating on a sparse set of the data. We prove that the resulting delayed acceptance MH is more efficient compared to that of Payne and Mallick (2015). The caveat of this approach is that the full data set needs to be evaluated in the second stage. We therefore propose to substitute this evaluation by an estimate and construct a state-dependent approximation thereof to use in the first stage. This results in an algorithm that (i) can use a smaller subsample m by leveraging on recent advances in Pseudo-Marginal MH (PMMH) and (ii) is provably within O(m −2 ) of the true posterior.

show abstract

“…Some authors proposed more general approximations of the likelihood ratio, leading to non-exact algorithms. References [ 78 , 79 , 80 , 81 ] proposed estimators based on subsampling when the data x are too large. Reference [ 82 ] proposed an estimator of the likelihood ratio when the likelihood has intractable constants, as in the exponential random graph model, and proved that, even if the resulting MCMC is inexact, it remains asymptotically close to the exact chain.…”

Section: Approximation In the Computationsmentioning

confidence: 99%