Linear Operators and Stochastic Partial Differential Equations in Gaussian Process Regression

Abstract

Abstract. In this paper we shall discuss an extension to Gaussian process (GP) regression models, where the measurements are modeled as linear functionals of the underlying GP and the estimation objective is a general linear operator of the process. We shall show how this framework can be used for modeling physical processes involved in measurement of the GP and for encoding physical prior information into regression models in form of stochastic partial differential equations (SPDE). We shall also illustrate the practical applicability of the theory in a simulated application.

Continuous-Space Gaussian Process Regression and Generalized Wiener Filtering with Application to Learning Curves

“…[9]). We could also relax the assumption about the measurements belonging to the same function space as the a priori Gaussian process, which would allow analysis of more general inverse problems.…”

“…The function is not observed directly, but instead, we measure a linear transformation of the signal, defined via the linear operator H x (cf. [9]), and the measurements y(x) are also corrupted by measurement noise e(x). Selection H x = 1 leads to the ordinary Gaussian process regression model…”

“…For notational convenience, we assume that both the function f (x) and the measured signal y(x) are scalar valued and zero mean. The full solution to the finite-measurement version of the generalized Gaussian process regression problem in Equation (1) was presented in article [9] and the solution corresponding to the case H x = 1 can be found in [1]. In image processing applications the including the operator H x into the model is very useful, because it can be used, for example, for modeling motion blurs or other linear degradations of images (cf.…”

“…We compare our learning curve approximation to several other estimates for ν = 3/2. In Figure 2 MC (n) is the 'true' generalization error curve computed by 100 independent Monte Carlo samples for each n with unit Gaussian input density; C (n) is the Gauss-Hermite learning curve approximation due to [9] using the 60th and 20th order Gauss-Hermite rule in the 1d and 2d examples, respectively; OV (n) is the Opper-Vivarelli bound [3]; D (n), UC (n), LC (n) are the bounds considered by Sollich and Halees [4]; MCU (n) is the 'true' generalization error the unit variance uniform input density; and, finally, (n) is the proposed learning curve approximation normalized to the same scale with the rest of the curves. The figures show that the approximation (n) underestimates the error for small n and overestimates it for large n when compared to the 'true' values MCU (n).…”

Gaussian Processes for Data Fulfilling Linear Differential Equations

“…In these earlier works GPs are usually referred to as Kriging and stationary covariance functions / kernels as covariograms. A number of more recent works from various fields [3,4,5] use the linear operator of the problem to obtain a new kernel function for the source field by applying it twice to a generic, usually squared exponential, kernel. In contrast to the present approach, that method is suited best for source fields that are non-vanishing across the whole domain.…”