Hilbert space methods for reduced-rank Gaussian process regression

Abstract

This paper proposes a novel scheme for reduced-rank Gaussian process regression. The method is based on an approximate series expansion of the covariance function in terms of an eigenfunction expansion of the Laplace operator in a compact subset of R d . On this approximate eigenbasis the eigenvalues of the covariance function can be expressed as simple functions of the spectral density of the Gaussian process, which allows the GP inference to be solved under a computational cost scaling as O(nm 2 ) (initial) and O(m 3 ) (hyperparameter learning) with m basis functions and n data points. The approach also allows for rigorous error analysis with Hilbert space theory, and we show that the approximation becomes exact when the size of the compact subset and the number of eigenfunctions go to infinity. The expansion generalizes to Hilbert spaces with an inner product which is defined as an integral over a specified input density. The method is compared to previously proposed methods theoretically and through empirical tests with simulated and real data.

Robust inference of two-dimensional strain fields from diffraction-based measurements

“…The squared-exponential covariance function can be approximated by a finite sum of m basis functions as in [19];…”

The Use of Gaussian Processes in System Identification

“…Ways of solving the computational complexity problem are, for example, sparse approximations using inducing points (Quiñonero-Candela and Rasmussen, 2005;Rasmussen and Williams, 2006;Titsias, 2009), approximating the problem with a discrete Gaussian random field model (Lindgren et al, 2011), or use of random or deterministic basis/spectral expansions (Quiñonero-Candela et al, 2010;Solin and Särkkä, 2018).…”

Scalable Magnetic Field SLAM in 3D Using Gaussian Process Maps

“…Extending previous work [9], we build the magnetic field map using Gaussian process regression [10] and incorporate physical knowledge about the magnetic field known from Maxwell's equations in the Gaussian process prior. To overcome issues with computational complexity, we use the rank-reduced approach first presented in [11]. This results in a representation that fits perfectly in a Rao-Blackwellised particle filter (RBPF) [12] which can be used both for building the map and for localisation in the map, resulting in a tractable localisation algorithm.…”

“…The approach presented in [11] relies on a basis function expansion on a specific domain where the required number of basis functions scales with the size of the domain. To allow for mapping large 3D areas, we propose a map representation using hexagonal block tiling, which aims at providing a very compact (in terms of required storage per volume) representation of the magnetic field map (all three vector field components and associated uncertainties).…”