Gaussian process regression with functional covariates and multivariate response

Abstract

Gaussian process regression (GPR) has been shown to be a powerful and effective nonparametric method for regression, classification and interpolation, due to many of its desirable properties. However, most GPR models consider univariate or multivariate covariates only. In this paper we extend the GPR models to cases where the covariates include both functional and multivariate variables and the response is multidimensional. The model naturally incorporates two different types of covariates: multivariate and functional, and the principal component analysis is used to de-correlate the multivariate response which avoids the widely recognised difficulty in the multi-output GPR models of formulating covariance functions which have to describe the correlations not only between data points but also between responses. The usefulness of the proposed method is demonstrated through a simulated example and two real data sets in chemometrics.

K-Nearest Neighbor Method with Principal Component Analysis for Functional Nonparametric Regression

“…Rinnan and Rinnan 21 analysed this data set originaly, after that 16 took a sample of these data and utilized Gaussian process regression with multivariate response on two components soil organic matter (SOM) and ergosterol concentration (EC). The soil data samples were obtained from a long-term field experiment at a subarctic fell in Abisko, northern Sweeden.…”

“…Chaouch and Laïb 15 explained the issue of multivariate response model from functional covariates based on the 𝐿1-median regression estimation approach. Wang and Chen 16 approached the Gaussian process regression with multivariate output and used principal component analysis to de-correlate multivariate response with functional and multivariate covariate variables. The nonparametric functional regression model for multivariate longitudinal data with multiple responses which is illustrated by different types of data for more detials see 17 .…”

An informative prior distribution on functions with application to functional regression

“…and 𝛽 = (𝛽 t , t ∈ T) is a centered, real-valued Gaussian process. Gaussian process regression has received considerable attention (see O'Hagan, 1978;Rasmussen & Williams, 2006 for scalar covariates and Shi & Choi, 2011;Wang, Chen, & Xu, 2017;Konzen, Cheng, & Shi, 2021 for functional covariates and a functional response) but the special case above (2) has been studied more as an inverse problem (Gugushvili, van der Vaart, & Yan, 2020;Knapik, van der Vaart, & van Zanten, 2011;Stuart, 2010). Note that in the latter case, 𝛽 is not defined as a process but as a random element of a function space with a Gaussian measure.…”

“…With such a prior, it can be seen (Section 2) that (1) is a special case of the Gaussian process regression model and can be written as follows: $$$$ {Y}_i=\mu +F\left({x}_i\right)+{\varepsilon}_i,\kern1em i=1,\dots, n,\kern0.5em $$$$ where the process $$$ F=\left(F(x),x\in {L}^1(T)\right) $$$ is defined by $$$$ F(x)={\int}_Tx(t){\beta}_t\mathrm{d}t,\kern0.5em $$$$ and $$$ \beta =\left({\beta}_t,t\in T\right) $$$ is a centered, real‐valued Gaussian process. Gaussian process regression has received considerable attention (see O'Hagan, 1978; Rasmussen & Williams, 2006 for scalar covariates and Shi & Choi, 2011; Wang, Chen, & Xu, 2017; Konzen, Cheng, & Shi, 2021 for functional covariates and a functional response) but the special case above (2) has been studied more as an inverse problem (Gugushvili, van der Vaart, & Yan, 2020; Knapik, van der Vaart, & van Zanten, 2011; Stuart, 2010). Note that in the latter case, $$$ \beta $$$ is not defined as a process but as a random element of a function space with a Gaussian measure.…”