Posterior inference for sparse hierarchical non-stationary models

Abstract

Gaussian processes are valuable tools for non-parametric modelling, where typically an assumption of stationarity is employed. While removing this assumption can improve prediction, fitting such models is challenging. In this work, hierarchical models are constructed based on Gaussian Markov random fields with stochastic spatially varying parameters. Importantly, this allows for non-stationarity while also addressing the computational burden through a sparse banded representation of the precision matrix. In this setting, efficient Markov chain Monte Carlo (MCMC) sampling is challenging due to the strong coupling a posteriori of the parameters and hyperparameters. We develop and compare three adaptive MCMC schemes and make use of banded matrix operations for faster inference. Furthermore, a novel extension to multi-dimensional settings is proposed through an additive structure that retains the flexibility and scalability of the model, while also inheriting interpretability from the additive approach. A thorough assessment of the efficiency and accuracy of the methods in nonstationary settings is presented for both simulated experiments and a computer emulation problem.

Physics-informed CoKriging: A Gaussian-process-regression-based multifidelity method for data-model convergence

“…A more general approach to GPR is to employ parameterized nonstationary covariance kernels. Nonstationary kernels can be obtained by modifying stationary covariance kernels, e.g., [41,27,32,37,4,28], or from neural networks with specific activation functions, e.g., [29,34], among other approaches. Many of these approaches assume a specific functional form for the correlation function, chosen according to expert knowledge.…”

Enhancing industrial X-ray tomography by data-centric statistical methods

“…These include, for example, deep Gaussian processes (Dunlop et al, 2018;Emzir et al, 2020), level-set methods (Dunlop et al, 2017), mixtures of compound Poisson processes and Gaussians (Hosseini, 2017), and stacked Matérn fields via stochastic partial differential equations (Roininen et al, 2019). The problem with hierarchical priors is that in the posteriors, the parameters and hyperparameters may become strongly coupled, which means that vanilla MCMC methods become problematic and, for example, reparameterizations are needed for sampling the posterior efficiently (Chada et al, 2019;Monterrubio-Gómez et al, 2020). In level-set methods, the number of levels is usually low because experiments have shown that the method deteriorates when the number of levels is increased.…”

Cluster, classify, regress: A general method for learning discontinuous functions

“…This example is grid-free, at least, and this feature will be desirable for high-dimensional problems of the type considered here. It will be interesting to compare this method to grid-based Bayesian approaches such as [11,24]. If the regressors are defined in terms of grids, for example θ l r are the coefficients of expansion in piecewise linear finite element nodal basis functions [38], then there are similarities between the methods.…”

“…It is clear that neither of the simple regression methods are able to cleanly recover the discontinuity. Now we consider slightly more complicated functions, following the recent work [24]. First, consider X = [−4, 10] and…”

“…The classical problems of machine learning are typically either of classification or regression type. Highly nonlinear and discontinuous functions are something in between, and their reconstruction has been the focus of much existing work in the literature [12,1,35,15,3,2,29,19], but it is still an active area [5,17,37,24,11]. In the present work, we propose a simple general framework which can be utilized to solve such problems, using as components the arsenal of available machine learning algorithms [25,6,13,30,16].…”