The Block-Poisson Estimator for Optimally Tuned Exact Subsampling MCMC

Quiroz, Matias; Tran, Minh‐Ngoc; Villani, Mattias; Kohn, Robert; Dang, Khue‐Dung

doi:10.1080/10618600.2021.1917420

Cited by 8 publications

(22 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, Vihola et al [2020] proposed an importance sampling correction of approximate MCMC to yield exact inference. While this could be applied to the scheme of Hensman et al [2015], instead, in the next section, we construct a pseudo-marginal sampler based on the block-Poisson estimator of Quiroz et al [2020]. Importantly, our scheme bypasses the problems of the approximate MCMC scheme, by providing both asymptotically exact inference for the optimal varitional posterior and reduced computational complexity.…”

Section: Variationally Sparse Gpsmentioning

confidence: 99%

“…Recently, Quiroz et al [2020] proposed combining an importance sampling sign correction [Lyne et al, 2015] with a product of Poisson estimators [Fearnhead et al, 2010] to derive a signed block-Poisson pseudo-marginal scheme for fast, exact inference in tall datasets. Their block-Poisson estimator is appealing for several reasons.…”

Section: Pm For Variationally Sparse Gpsmentioning

confidence: 99%

“…In this work, we build on Quiroz et al [2020] to construct a signed block-Poisson PM scheme for variationally sparse GPs, that is both computationally efficient for large datasets and offers asymptotically exact inference for the lowdimensional variational posterior. This is accomplished through a variant of the PM scheme that employs a doubly stochastic estimator for the intractable exponentiated expected log-likelihood, by also data subsampling.…”

Section: Pm For Variationally Sparse Gpsmentioning

confidence: 99%

“…that is also computationally efficient for large sample sizes. To do so, we follow Quiroz et al [2020] by employing subsampling for computational efficiency and control variates to reduce the variance of our estimator. In addition, we employ a further layer of stochasticity in our estimator to deal with the intractable expectation.…”

Section: Doubly Stochastic Block-poisson Estimatormentioning

confidence: 99%

“…We propose an alternative approach to bypass this problem by replacing the intractable likelihood evaluation with a computationally cheap unbiased estimate based on the block-Poisson estimator introduced by Quiroz et al [2020]. Our proposed framework offers asymptotically exact inference for the low-dimensional variational posterior through a pseudo-marginal scheme as well as computational gains through doubly stochastic estimators for the intractable likelihood evaluations and large datasets.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

On MCMC for variationally sparse Gaussian processes: A pseudo-marginal approach

Monterrubio-Gómez,

Wade

2021

Preprint

View full text Add to dashboard Cite

Gaussian processes (GPs) are frequently used in machine learning and statistics to construct powerful models. However, when employing GPs in practice, important considerations must be made, regarding the high computational burden, approximation of the posterior, choice of the covariance function and inference of its hyperparmeters. To address these issues, Hensman et al. [2015] combine variationally sparse GPs with Markov chain Monte Carlo (MCMC) to derive a scalable, flexible and general framework for GP models. Nevertheless, the resulting approach requires intractable likelihood evaluations for many observation models. To bypass this problem, we propose a pseudomarginal (PM) scheme that offers asymptotically exact inference as well as computational gains through doubly stochastic estimators for the intractable likelihood and large datasets. In complex models, the advantages of the PM scheme are particularly evident, and we demonstrate this on a twolevel GP regression model with a nonparametric covariance function to capture non-stationarity.

show abstract

Section: Variationally Sparse Gpsmentioning

confidence: 99%