Forward Adaptive Transfer of Gaussian Process Regression

Wagle, Neeti; Frew, Eric W.

doi:10.2514/1.i010437

Cited by 18 publications

(15 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, Wei et al [6] investigate the extension of T λ into the multiple-source transfer scenario and demonstrate a similar pathology. Wagle et al [27] further incorporate T λ with cross-domain matrices [28] to construct a new PSD transfer kernel. In [29], the authors propose a fast and scalable GP model based on T λ for the large source scenario.…”

Section: Related Workmentioning

confidence: 99%

Easy-But-Effective Domain Sub-Similarity Learning for Transfer Regression

Wei

Sagarna

et al. 2022

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

Transfer covariance function, which can model domain similarity and adaptively control the knowledge transfer across domains, is widely used in transfer learning. In this paper, we concentrate on Gaussian process (GP) models using a transfer covariance function for regression problems in a black-box learning scenario. Precisely, we investigate a family of rather general transfer covariance functions, T * , that can model the heterogeneous sub-similarities of domains through multiple kernel learning. A necessary and sufficient condition to obtain valid GPs using T * (GP T * ) for any data is given. This condition becomes specially handy for practical applications as (i) it enables semantic interpretations of the sub-similarities and (ii) it can readily be used for model learning. In particular, we propose a computationally inexpensive model learning rule that can explicitly capture different sub-similarities of domains. We propose two instantiations of GP T * , one with a set of predefined constant base kernels and one with a set of learnable parametric base kernels. Extensive experiments on 36 synthetic transfer tasks and 12 real-world transfer tasks demonstrate the effectiveness of GP T * on the sub-similarity capture and the transfer performance.

show abstract

Section: Related Workmentioning

confidence: 99%

Easy-But-Effective Domain Sub-Similarity Learning for Transfer Regression

Wei

Sagarna

et al. 2022

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

show abstract

“…In [6], Cao et al propose an adaptive transfer model that utilizes a single similarity coefficient to reweight the covariance of the source and target domains, i.e., TGP. In [26], Wagle et al follow [6], and improve the transfer model by further distinguishing the source-to-source covariance from the target-to-target covariance. Wei et al [32] investigates an extension of [6] into the multiple-source transfer scenario and demonstrate a pathology.…”

Section: Related Workmentioning

confidence: 99%

“…Although the proposed mTGP is based on a specific explicit relatedness modelling, i.e., TGP [6], the idea of succinct manifold learning, specifically Φ(•) , is applicable to other similar methods with explicit relatedness representation, e.g., [26,31,32]. The reason why we choose to build on TGP is because it is the very first fundamental method for explicit modelling of relatedness, and the following works [26,31,32] are based on it. We then emphasize the importance of the latent manifold learning to the transfer performance by comparing mTGP and TGP that directly works in the original input feature space.…”

Section: Related Workmentioning

confidence: 99%

Succinct Adaptive Manifold Transfer

Wei

et al. 2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

Capturing the relatedness of different domains is a key challenge in transferring knowledge across domains. In this paper, we propose an effective and efficient Gaussian process (GP) modelling framework, mTGP , that can explicitly model domain relatedness and adaptively control the space as well as the strength of knowledge transfer. mTGP takes both the discrepancy of input feature space and the discrepancy of predictive function into account in the transfer procedure. Specifically, mTGP adaptively selects a good latent manifold shared by different domains, and utilizes a parametric similarity coefficient to measure the predictive function covariance of different domains in this manifold. The latent shared manifold and the similarity coefficient are jointly learned in a coupled manner. By doing so, mTGP maximizes the strength of the shared knowledge transfer by choosing the transfer space with the best transfer capacity. More importantly, mTGP exploits a succinct and computationally efficient manifold learning approach so that it can be well trained with scarce target training data. Extensive experimental studies using 36 synthetic transfer tasks and 10 real-world transfer tasks show the effectiveness of mTGP on capturing the relatedness and the transfer adaptiveness. CCS CONCEPTS • Computing methodologies → Transfer learning.

show abstract

“…Surrogate-assisted modeling and optimization have been extensively deployed to facilitate modern aeroengine design [1,2,3,4] due to the representational capability of complex features. Among current surrogates (also known as machine learning models), as a non-parametric Bayesian model, Gaussian process (GP) [5] (also known as Kriging or emulator), has gained popularity.…”

Section: Introductionmentioning

confidence: 99%

Understanding and comparing scalable Gaussian process regression for big data

Liu

Cai

Ong

et al. 2019

Knowledge-Based Systems

View full text Add to dashboard Cite

As a non-parametric Bayesian model which produces informative predictive distribution, Gaussian process (GP) has been widely used in various fields, like regression, classification and optimization. The cubic complexity of standard GP however leads to poor scalability, which poses challenges in the era of big data. Hence, various scalable GPs have been developed in the literature in order to improve the scalability while retaining desirable prediction accuracy. This paper devotes to investigating the methodological characteristics and performance of representative global and local scalable GPs including sparse approximations and local aggregations from four main perspectives: scalability, capability, controllability and robustness. The numerical experiments on two toy examples and five real-world datasets with up to 250K points offer the following findings. In terms of scalability, most of the scalable GPs own a time complexity that is linear to the training size. In terms of capability, the sparse approximations capture the long-term spatial correlations, the local aggregations capture the local patterns but suffer from over-fitting in some scenarios. In terms of controllability, we could improve the performance of sparse approximations by simply increasing the inducing size. But this is not the case for local aggregations. In terms of robustness, local aggregations are robust to various initializations of hyperparameters due to the local attention mechanism. Finally, we highlight that the proper hybrid of global and local scalable GPs may be a promising way to improve both the model capability and scalability for big data.

show abstract

Forward Adaptive Transfer of Gaussian Process Regression

Cited by 18 publications

References 16 publications

Easy-But-Effective Domain Sub-Similarity Learning for Transfer Regression

Easy-But-Effective Domain Sub-Similarity Learning for Transfer Regression

Succinct Adaptive Manifold Transfer

Understanding and comparing scalable Gaussian process regression for big data

Contact Info

Product

Resources

About