Maximum likelihood estimation for Gaussian processes under inequality constraints

Abstract

We consider covariance parameter estimation for a Gaussian process under inequality constraints (boundedness, monotonicity or convexity) in fixed-domain asymptotics. We address the estimation of the variance parameter and the estimation of the microergodic parameter of the Matérn and Wendland covariance functions. First, we show that the (unconstrained) maximum likelihood estimator has the same asymptotic distribution, unconditionally and conditionally to the fact that the Gaussian process satisfies the inequality constraints. Then, we study the recently suggested constrained maximum likelihood estimator. We show that it has the same asymptotic distribution as the (unconstrained) maximum likelihood estimator. In addition, we show in simulations that the constrained maximum likelihood estimator is generally more accurate on finite samples. Finally, we provide extensions to prediction and to noisy observations. It is still challenging to obtain results on maximum likelihood estimation of microergodic parameters that would hold for very general classes of covariance functions. Nevertheless, significant contributions have been made for specific types of covariance functions. In particular, when considering the isotropic Matérn family of covariance functions, for input space dimension d = 1, 2, 3, a reparameterized quantity obtained from the variance and correlation length parameters is microergodic (Zhang, 2004). It has been shown in (Kaufman and Shaby, 2013), from previous results in (Du et al., 2009) and(Wang andLoh, 2011), that the maximum likelihood estimator of this microergodic parameter is consistent and asymptotically Gaussian distributed. Anterior results on the exponential covariance function have been also obtained in (Ying, 1991(Ying, , 1993.In this paper, we shall consider the situation where the trajectories of the Gaussian process are known to satisfy either boundedness, monotonicity or convexity constraints. Indeed, Gaussian processes with inequality constraints provide suitable regression models in application fields such as computer networking (monotonicity) (Golchi et al., 2015), social system analysis (monotonicity) (Riihimäki and Vehtari, 2010) and econometrics (monotonicity or positivity) (Cousin et al., 2016). Furthermore, it has been shown that taking the constraints into account may considerably improve the predictions and the predictive intervals for the Gaussian process (Da Veiga and Marrel, 2012;Golchi et al., 2015;Riihimäki and Vehtari, 2010).Recently, a constrained maximum likelihood estimator (cMLE) for the covariance parameters has been suggested in . Contrary, to the (unconstrained) maximum likelihood estimator (MLE) discussed above, the cMLE explicitly takes into account the additional information brought by the inequality constraints. In , it is shown, essentially, that the consistency of the MLE implies the consistency of the cMLE under boundedness, monotonicity or convexity constraints.The aim of this paper is to study the asymptotic conditional distributions of the MLE and the cM...

Maximum Likelihood Estimation and Uncertainty Quantification for Gaussian Process Approximation of Deterministic Functions

“…For other work related to GP misspecification and kernel parameter estimation in a variety of settings, see Stein (1993), Bachoc (2013), Bachoc, Lagnoux, and Nguyen (2017), Bachoc (2017), Lopera-L\' opez (2019), andTeckentrup (2019).…”

Tensor Basis Gaussian Process Models of Hyperelastic Materials

“…This approach has the advantages of avoiding the burden of a large training set that comes with a neural network model, and the inexact satisfaction of constraints that come with penalization of constraints in the loss function. There has been significant interest in the incorporation of constraints into Gaussian process regression (GPR) models recently (Bachoc et al, 2019;Da Veiga and Marrel, 2012;Jensen et al, 2013;López-Lopera et al, 2018;Raissi et al, 2017;Riihimäki and Vehtari, 2010;Solak et al, 2003;Yang et al, 2018). Many of these approaches leverage the analytic formulation of the GP to incorporate constraints through the likelihood function or i.e.…”

Gaussian field on the symmetric group: Prediction and learning

“…We can then provide the best LHD of EI with the Spearman distance. This LHD is given by the permutations σ 2 = (5, 2, 1, 7, 6, 3,4,8,11,13,12,9,10,14,15), 6,1,8,4,9,15,7,12,5,13,10,2,11,14). To conclude, the kernels on permutations provided in Section 2 enable us to use EI that gives much better results than simulated annealing or random sampling to find the best LHD.…”

“…Nevertheless, remark that f is a positive function, whereas a Gaussian process realization can take negative values. In this case, different options are possible: firstly, we can ignore the information of the inequality constraint; secondly, we can use Gaussian process under inequality constraints (see [6]); thirdly, we can use a transformation of the function to remove the inequality constraint. We choose here the third strategy and we model log(f ) by a Gaussian process realization.…”

“…Remark 1. Since S N is a finite discrete space, remark that the Reproducible Kernel Hilbert Space (RKHS) of a kernel K is defined by the set of the functions of the form (6), and the universality of the kernel K is equivalent to the equality of its RKHS with the set of the functions from S N to R. This is, in turn, equivalent to the fact that K is strictly positive definite.…”