Learning Nonparametric Volterra Kernels with Gaussian Processes

Abstract

This paper introduces a method for the nonparametric Bayesian learning of nonlinear operators, through the use of the Volterra series with kernels represented using Gaussian processes (GPs), which we term the nonparametric Volterra kernels model (NVKM). When the input function to the operator is unobserved and has a GP prior, the NVKM constitutes a powerful method for both single and multiple output regression, and can be viewed as a nonlinear and nonparametric latent force model. When the input function is observed, the NVKM can be used to perform Bayesian system identification. We use recent advances in efficient sampling of explicit functions from GPs to map process realisations through the Volterra series without resorting to numerical integration, allowing scalability through doubly stochastic variational inference, and avoiding the need for Gaussian approximations of the output processes. We demonstrate the performance of the model for both multiple output regression and system identification using standard benchmarks.

A spectrum of physics-informed Gaussian processes for regression in engineering

“…In Pillonetto et al (2014) a review is provided in this context, where the focus is on the derivation of a covariance function that will act as an optimal regularizer for the learning of linear dynamical system parameters. Within the ML community, researchers attempting to develop more generic technology also look to physical systems to provide covariance functions useful for a broad class of problems (Higdon, 2002;Boyle and Frean, 2005;Alvarez et al, 2009;Wilson and Adams, 2013;Tobar et al, 2015;Parra and Tobar, 2017;Van der Wilk et al, 2017;McDonald and Álvarez, 2021;Ross et al, 2021). These examples present very flexible models that are able to perform very well for a variety of different tasks.…”