Parallel Local Approximation MCMC for Expensive Models

Abstract

Performing Bayesian inference via Markov chain Monte Carlo (MCMC) can be exceedingly expensive when posterior evaluations invoke the evaluation of a computationally expensive model, such as a system of partial differential equations. In recent work [10], we described a framework for constructing and refining local approximations of such models during an MCMC simulation. These posterior-adapted approximations harness regularity of the model to reduce the computational cost of inference while preserving asymptotic exactness of the Markov chain. Here we describe two extensions of that work. First, we prove that samplers running in parallel can collaboratively construct a shared posterior approximation while ensuring ergodicity of each associated chain, providing a novel opportunity for exploiting parallel computation in MCMC. Second, focusing on the Metropolis-adjusted Langevin algorithm, we describe how a proposal distribution can successfully employ gradients and other relevant information extracted from the approximation. We investigate the practical performance of our approach using two challenging inference problems, the first in subsurface hydrology and the second in glaciology. Using local approximations constructed via parallel chains, we successfully reduce the run time needed to characterize the posterior distributions in these problems from days to hours and from months to days, respectively, dramatically improving the tractability of Bayesian inference.

Uncertainty quantification of the multi-centennial response of the Antarctic ice sheet to climate change

“…Schlegel et al (2018) found that grounding-line retreat is most significant in the Amundsen Sea sector under generalised ocean warming experiments for the Antarctic ice sheet, but, after calibrating sub-shelf melt rates with bounds that vary region by region and are assigned values deduced from the literature and model sensitivity studies, they found that the western Ronne basin has the greater sensitivity. These discrepancies between our findings and those of Cornford et al (2015), Ritz et al (2015) and Schlegel et al (2018) may be explained by our ocean model which may overestimate sub-shelf melting underneath the Ross ice shelf and underestimate ocean circu- Figure 13. Same as Fig.…”

“…Consistent with our results, Golledge et al (2015), with a 10 km resolution model, projected that grounding-line retreat is most significant in the Siple Coast region. However, Ritz et al (2015), based on the probability of MISI onset, as well as Cornford et al (2015), with a sub-kilometre resolution around the grounding line, projected that grounding-line retreat is most significant in the Amundsen Sea sector. Schlegel et al (2018) found that grounding-line retreat is most significant in the Amundsen Sea sector under generalised ocean warming experiments for the Antarctic ice sheet, but, after calibrating sub-shelf melt rates with bounds that vary region by region and are assigned values deduced from the literature and model sensitivity studies, they found that the western Ronne basin has the greater sensitivity.…”

Efficient Bayesian inference for large chaotic dynamical systems

“…The LA-MCMC method (Conrad et al, 2016(Conrad et al, , 2018 approximates the computationally expensive loglikelihood function using local polynomial regression. In this method, the MCMC sampler directly uses 95 the approximation of the log-likelihood to construct proposals and evaluate the Metropolis acceptance probability.…”

“…There are many potential directions for extension of this work. First, it should be feasible to run parallel LA-MCMC chains that share model evaluations in a single evaluated set; doing so can accelerate the con-470 struction of accurate local surrogate models, as demonstrated in Conrad et al (2018), and is a useful way of harnessing parallel computational resources within surrogate-based MCMC. Extending this approach to higher-dimensional parameters is also of interest.…”

“…Extending this approach to higher-dimensional parameters is also of interest. While LA-MCMC has been successfully applied to chains of dimension up to q = 12 (Conrad et al, 2018), future work should explore sparsity and other truncations of the local polynomial approximation to improve scaling with dimension. From the CIL per-475 spective, calibrating more complex models, such as weather models, often requires choosing the part of the state vector from which the feature vectors are computed.…”