Asymptotic analysis of the learning curve for Gaussian process regression

Abstract

This paper deals with the learning curve in a Gaussian process regression framework. The learning curve describes the generalization error of the Gaussian process used for the regression. The main result is the proof of a theorem giving the generalization error for a large class of correlation kernels and for any dimension when the number of observations is large. From this theorem, we can deduce the asymptotic behavior of the generalization error when the observation error is small. The presented proof generalizes previous ones that were limited to special kernels or to small dimensions (one or two). The theoretical results are applied to a nuclear safety problem. Keywords Gaussian process regression • Asymptotic mean squared error • Learning curves • Generalization error • Convergence rate 1 Introduction Gaussian process regression is a useful tool to approximate an objective function given some of its observations (Laslett 1994). It has originally been used in geostatistics to interpolate a random field at unobserved locations (Wackernagel 2003; Berger et al. 2001; Gneiting et al. 2010), it has been developed in many areas such as environmental and atmospheric sciences.

Sequential design strategies for mean response surface metamodeling via stochastic kriging with adaptive exploration and exploitation

“…We note that the integrated mean squared error (IMSE) based criteria are often used for constructing experimental designs such that the resulting metamodel can achieve satisfactory predictive performance across the design space. For kriging prediction with nugget effect, Gratiet and Garnier (2015) provide theoretical results on the asymptotic values of pointwise MSE and IMSE and obtain the convergence rates of IMSE for selected kernels as the number of design points increases to infinity; they further discuss how the theoretical results can be used to determine the total budget required to achieve a pre-specified accuracy level in terms of IMSE. The differences between their work and our results given in Theorem 1 lie on the following aspects.…”

“…With respect to metamodeling for mean response prediction in the stochastic simulation setting, some in-depth work has been conducted regarding asymptotic properties of kriging prediction with nugget and the implications on nonsequential experimental designs with a large simulation budget are provided (Gratiet and Garnier, 2015). However, a systematic account of sequential designs with a fixed simulation budget has yet to be established, despite the earlier efforts by van Beers and Kleijnen (2008), Ng and Yin (2012), Ajdari and Mahlooji (2014) and Mehdad and Kleijnen (2015).…”

Asymptotic analysis of covariance parameter estimation for Gaussian processes in the misspecified case

“…The case where Y is observed exactly is treated by this framework by letting δ 0 = 0. Otherwise, letting δ 0 > 0 can correspond for instance to measure errors (Bachoc et al, 2014) or to Monte Carlo computer experiments (Le Gratiet and Garnier, 2014). Note also that the case of a Gaussian process with discontinuous covariance function at 0 (nugget effect) is mathematically equivalent to this framework if the observation points X 1 , ..., X n are two by two distinct.…”

A Divide-and-Conquer Bayesian Approach to Large-Scale Kriging

“…As a by-product of the proof of Theorem 3.1, the upper bound for var DISK can be used to show that the integrated predictive variance of GP decreases to zero as the subset sample size m → ∞ for various types of covariance kernels. A closely related work by Gratiet and Garnier (2015) studies the asymptotic behavior for the mean squared error of GP, but unrealistically assumes that the error variance increases with the sample size, which prevents their predictive variance of GP from converging to zero.…”