Semi-Exact Control Functionals From Sard's Method

Abstract

This paper focuses on the numerical computation of posterior expected quantities of interest, where existing approaches based on ergodic averages are gated by the asymptotic variance of the integrand. To address this challenge, a novel variance reduction technique is proposed, based on Sard's approach to numerical integration and the control functional method. The use of Sard's approach ensures that our control functionals are exact on all polynomials up to a fixed degree in the Bernstein-von-Mises limit, so that the reduced variance estimator approximates the behaviour of a polynomiallyexact (e.g. Gaussian) cubature method. The proposed estimator has reduced mean square error compared to its competitors, and is illustrated on several Bayesian inference examples. All methods used in this paper are available in the R package ZVCV.

Post-Processing of MCMC

“…is satisfied for some C ∈ R, some δ > d − 1, and all x ∈ R d outside of a bounded set, then the Stein identity is satisfied (South et al 2020).…”

“…To address the poor performance of CF relative to CV in the highdimensional context, South et al (2020) generalised the approaches discussed in Section 3.2.1 and Section 3.2.2, to consider functional approximations of the form…”

“…The performance of this hybrid approach depends on how the parameters θ are selected. Following Sard (1949), South et al (2020) propose to select θ such that the following properties are satisfied:…”

Positively Weighted Kernel Quadrature via Subsampling

“…These two papers also treat the more general quadrature problem that includes a weight function g ∈ L 2 (µ), i.e., approximating X f (x)g(x) dµ(x) for f ∈ H k ⊂ L 2 (µ), which we do not discuss in this article. Finally, we emphasize that the literature on kernel quadrature is vast and the above distinction is only a rough dichotomy: in addition to kernel herding or sample optimization, the literature includes their hybrid (Briol et al, 2015), kernel quadrature combined with control variate and/or Sard's method (Oates et al, 2017;Karvonen et al, 2018;South et al, 2020), and convergence analysis for the case when the RKHS is miss-specified (Kanagawa et al, 2016(Kanagawa et al, , 2020.…”

“…Sard's method (Sard, 1949;Larkin, 1970) for constructing numerical integration rules uses the n degree of freedom (of choosing weights in our setting) separately; m (≤ n) for exactness over a certain m-dimensional space of test functions, and the remaining n − m for minimizing an error criterion such as the worst-case error. In the context of kernel quadrature, one way to use Sard's method with exactness over F (an m-dimensional space of test functions) is as follows (Karvonen et al, 2018;South et al, 2020):…”