Recursive estimation for sparse Gaussian process regression

Schürch, Manuel; Azzimonti, Dario; Benavoli, Alessio; Zaffalon, Marco

doi:10.1016/j.automatica.2020.109127

Cited by 27 publications

(26 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As future work, we plan to investigate the possibility of using inducing points, as for sparse GPs (Quiñonero-Candela & Rasmussen, 2005;Snelson & Ghahramani, 2006;Titsias, 2009;Hensman et al, 2013;Hernández-Lobato & Hernández-Lobato, 2016;Bauer et al, 2016;Schuerch et al, 2020), to reduce the computational load for matrix operations (complexity O(n 3 ) with storage demands of O(n 2 ) ). We also plan to derive tighter approximations of the marginal likelihood.…”

Section: Discussionmentioning

confidence: 99%

A unified framework for closed-form nonparametric regression, classification, preference and mixed problems with Skew Gaussian Processes

2021

Self Cite

View full text Add to dashboard Cite

Skew-Gaussian Processes (SkewGPs) extend the multivariate Unified Skew-Normal distributions over finite dimensional vectors to distribution over functions. SkewGPs are more general and flexible than Gaussian processes, as SkewGPs may also represent asymmetric distributions. In a recent contribution, we showed that SkewGP and probit likelihood are conjugate, which allows us to compute the exact posterior for non-parametric binary classification and preference learning. In this paper, we generalize previous results and we prove that SkewGP is conjugate with both the normal and affine probit likelihood, and more in general, with their product. This allows us to (i) handle classification, preference, numeric and ordinal regression, and mixed problems in a unified framework; (ii) derive closed-form expression for the corresponding posterior distributions. We show empirically that the proposed framework based on SkewGP provides better performance than Gaussian processes in active learning and Bayesian (constrained) optimization. These two tasks are fundamental for design of experiments and in Data Science.

show abstract

Section: Discussionmentioning

confidence: 99%

A unified framework for closed-form nonparametric regression, classification, preference and mixed problems with Skew Gaussian Processes

2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the numerical experiments, we have used a full GP whose computational load grows cubically as the size of the training set increases. Sparse GPs can be employed to address this issue [Bauer et al 2016;Hensman et al 2013;Hernández-Lobato and Hernández-Lobato 2016;Quiñonero-Candela and Rasmussen 2005;Schuerch et al 2020;Snelson and Ghahramani 2006;Titsias 2009] when it is necessary to sample thousands of samples.…”

Section: Discussionmentioning

confidence: 99%

Gaussian Processes to speed up MCMC with automatic exploratory-exploitation effect

Benavoli,

Wyse,

White

2021

Preprint

Self Cite

View full text Add to dashboard Cite

“…This results in an ever growing set of retained test points. Alternatively, one could implement Bayesian filtering techniques explained in [48,49] to combine the previously calculated posterior with a new one based on only one data point and a new set of test points.…”

Section: Discussionmentioning

confidence: 99%

Feasibility of Kd-Trees in Gaussian Process Regression to Partition Test Points in High Resolution Input Space

et al. 2020

View full text Add to dashboard Cite

Bayesian inference using Gaussian processes on large datasets have been studied extensively over the past few years. However, little attention has been given on how to apply these on a high resolution input space. By approximating the set of test points (where we want to make predictions, not the set of training points in the dataset) by a kd-tree, a multi-resolution data structure arises that allows for considerable gains in performance and memory usage without a significant loss of accuracy. In this paper, we study the feasibility and efficiency of constructing and using such a kd-tree in Gaussian process regression. We propose a cut-off rule that is easy to interpret and to tune. We show our findings on generated toy data in a 3D point cloud and a simulated 2D vibrometry example. This survey is beneficial for researchers that are working on a high resolution input space. The kd-tree approximation outperforms the naïve Gaussian process implementation in all experiments.

show abstract

Recursive estimation for sparse Gaussian process regression

Cited by 27 publications

References 19 publications

A unified framework for closed-form nonparametric regression, classification, preference and mixed problems with Skew Gaussian Processes

A unified framework for closed-form nonparametric regression, classification, preference and mixed problems with Skew Gaussian Processes

Gaussian Processes to speed up MCMC with automatic exploratory-exploitation effect

Feasibility of Kd-Trees in Gaussian Process Regression to Partition Test Points in High Resolution Input Space

Contact Info

Product

Resources

About