The reproducing Stein kernel approach for post-hoc corrected sampling

Abstract

Stein importance sampling [42] is a widely applicable technique based on kernelized Stein discrepancy [43], which corrects the output of approximate sampling algorithms by reweighting the empirical distribution of the samples. A general analysis of this technique is conducted for the previously unconsidered setting where samples are obtained via the simulation of a Markov chain, and applies to an arbitrary underlying Polish space. We prove that Stein importance sampling yields consistent estimators for quantities related to a target distribution of interest by using samples obtained from a geometrically ergodic Markov chain with a possibly unknown invariant measure that differs from the desired target. The approach is shown to be valid under conditions that are satisfied for a large number of unadjusted samplers, and is capable of retaining consistency when data subsampling is used. Along the way, a universal theory of reproducing Stein kernels is established, which enables the construction of kernelized Stein discrepancy on general Polish spaces, and provides sufficient conditions for kernels to be convergence-determining on such spaces. These results are of independent interest for the development of future methodology based on kernelized Stein discrepancies.

Optimal Thinning of MCMC Output

“…Liu and Lee (2017) considered the use of kernel Stein discrepancy to optimally weight an arbitrary set (X i ) n i=1 ⊂ R d of states in an manner loosely analogous to importance sampling, at a computational cost of O(n 3 ). The combined effect of applying the approach of Liu and Lee (2017) to MCMC output was analysed in Hodgkinson et al (2020), who established situations in which the overall procedure will be consistent. The present paper differs from Liu and Lee (2017) and Hodgkinson et al (2020) in that we attempt compression, rather than weighting of the MCMC output.…”

Post-Processing of MCMC

“…This result, and the related results in Liu & Lee (2017), Hodgkinson et al (2020), weaken or remove the requirement to design Markov chains that are exactly P -invariant. See also Gramacy et al (2010), Radivojević & Akhmatskaya (2020).…”

Semi-Exact Control Functionals From Sard's Method

“…The requirement in Lemma 4 for the x x x (i) to be distinct precludes, for example, the direct use of Metropolis-Hastings output. However, as emphasized in Oates et al (2017) for control functionals and studied further in Liu and Lee (2017); Hodgkinson et al (2020), the consistency of I SECF does not require that the Markov chain is p-invariant. It is therefore trivial to, for example, filter out duplicate states from Metropolis-Hastings output.…”

“…Note that our choice of Stein operator differs to that in Chwialkowski et al (2016) and Liu et al (2016). There has been substantial recent research into the use of kernel Stein discrepancies for assessing algorithm performance in the Bayesian computational context (Gorham and Mackey, 2017;Chen et al, 2018Chen et al, , 2019Singhal et al, 2019;Hodgkinson et al, 2020) and one can also exploit this discrepancy as a diagnostic for the performance of the semi-exact control functional. The second quantity |f | k 0 ,F in the bound can be approximated by q q q q q q q q q q 0.03 0.10 0.30 1.00 100 300 1000 m q q q q mean absolute error mean upper bound…”