Pairwise Meta-Modeling of Multivariate Output Computer Models Using Nonseparable Covariance Function

Li, Yongxiang; Qiang, Zexuan

doi:10.1080/00401706.2015.1079244

Cited by 16 publications

(8 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides that, the computation is plagued by numerical issues associated with inverting the 990 × 990 covariance matrix at each iteration of the search algorithm. Similar results have also been shown in Li and Zhou (2016); Li et al (2018);Shi and Choi (2011), where MGP models tend to loose accuracy in a high dimensional parameter space. This issue is also faced in the MGCP-I, which is not able to address the large parameter space challenge, despite tackling the computational complexity where it only requires the inversion of 20 × 20 covariance matrices.…”

Section: Setting IIsupporting

confidence: 85%

“…Even for a moderate scale case where N = 30 and D = 1, we are required to estimate 1770 parameters under a non-convex setting. In another case, considering the more restrictive approach in Li and Zhou (2016) and Li et al (2018) where Q = 1 and all outputs possess the same noise, i.e. σ = σ 1 =, .., = σ N , the number of parameters still scales as N (N + 2D + 1)/2 + 1, therefore for N = 30 and D = 1 we are estimating 991 parameters.…”

Section: General Multivariate Gp Settingmentioning

confidence: 99%

“…In this article we propose a pairwise distributed estimation scheme motivated by both the distributed estimation literature for univariate GP models (Tresp, 2000;Deisenroth and Ng, 2015) and pairwise modeling of longitudinal profiles (Fieuws and Verbeke, 2006;Li and Zhou, 2016). The proposed approach is based on distributing MGCP estimation through bivariate GP submodels which are individually estimated.…”

Section: Motivation: Pairwise Estimation and The Precision Matrixmentioning

confidence: 99%

“…The MAE values are denoted as prediction errors in all boxplots. Kernel parameters are obtained through minimizing the negative log-likelihood using a scaled conjugate gradient algorithm (Li and Zhou (2016); Álvarez and Lawrence (2011); Rasmussen (2004)). For the MGCP-RD, we distribute the computations over only two systems, where each system was responsible for sequentially fitting (N − 1)/2 of the bivariate models.…”

Section: General Settingsmentioning

confidence: 99%

“…All computations are done on R-3.2.2 in a 64-bit Windows 7 setting. Further, in our simulation studies, we benchmark our method with four other reference methods for comparison: 1) The individual GP established using a CP, denoted as GCP, where the test function is fitted separately (Rasmussen, 2004); 2) The full MGCP model, denoted as MGCP, described in Section 3.2; 3) The inducing variable approximation, denoted as MGCP-I, which tackles the computational complexity challenge ( Álvarez and Lawrence, 2011;Alvarez et al, 2012); 4) The pairwise model for longitudinal profiles, denoted as MGCP-P (Fieuws and Verbeke, 2006;Li and Zhou, 2016;Li et al, 2018). To provide consistent results we utilize the Gaussian kernel in Section 4.1 for each of the benchmarked methods.…”

Section: General Settingsmentioning

confidence: 99%

See 4 more Smart Citations

Minimizing Negative Transfer of Knowledge in Multivariate Gaussian Processes: A Scalable and Regularized Approach

Kontar¹,

Raskutti²,

Zhou³

2021

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Recently there has been an increasing interest in the multivariate Gaussian process (MGP) which extends the Gaussian process (GP) to deal with multiple outputs. One approach to construct the MGP and account for non-trivial commonalities amongst outputs employs a convolution process (CP). The CP is based on the idea of sharing latent functions across several convolutions. Despite the elegance of the CP construction, it provides new challenges that need yet to be tackled. First, even with a moderate number of outputs, model building is extremely prohibitive due to the huge increase in computational demands and number of parameters to be estimated. Second, the negative transfer of knowledge may occur when some outputs do not share commonalities. In this paper we address these issues. We propose a regularized pairwise modeling approach for the MGP established using CP. The key feature of our approach is to distribute the estimation of the full multivariate model into a group of bivariate GPs which are individually built. Interestingly pairwise modeling turns out to possess unique characteristics, which allows us to tackle the challenge of negative transfer through penalizing the latent function that facilitates information sharing in each bivariate model. Predictions are then made through combining predictions from the bivariate models within a Bayesian framework. The proposed method has excellent scalability when the number of outputs is large and minimizes the negative transfer of knowledge between uncorrelated outputs. Statistical guarantees for the proposed method are studied and its advantageous features are demonstrated through numerical studies.

show abstract

Section: Setting IIsupporting

confidence: 85%

Section: General Multivariate Gp Settingmentioning

confidence: 99%