On the impact of prior distributions on efficiency of sparse Gaussian process regression

Esmaeilbeigi, Mohsen; Chatrabgoun, Omid; Daneshkhah, Alireza; Shafa, Maryam

doi:10.1007/s00366-022-01686-7

Cited by 4 publications

(2 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This limitation can be overcome using sparse approximation methods [26][27][28]. Using these methods, an approximation can be constructed based on a small set of m (≪ n) auxiliary variables, also known as inducing variables, that allow the reduction of the time complexity from O(n 3 ) to O(nm 2 ).…”

Section: Learning Of the Bayesian Gaussian Process Latent Variable Mo...mentioning

confidence: 99%

Uncertainty Quantification of PDE-based models using Deep Gaussian Processes and Variational Bayesian Inference

Fanous,

Chatrabgoun,

Esmaeilbeigi

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

Uncertainty quantification (UQ) and propagation (UP) in complex multi-scale physical models are challenging problems due to the high-dimensionality of the inputs and the outputs for such models and the corresponding computational complexity. These challenges could be tackled using Gaussian process (GP) models by constructing a surrogate response surface. This response surface is an efficient approximation of the underlying expensive model which can be then used for UQ and UP. However, the standard GP model is hampered by various issues such as the curse of dimensionality, difficulties in capturing the non-Gaussian complex nature of the input/output relation, and modelling discontinuities of 1 model outputs. To overcome these issues, this work aims to use a non-linear GP model, known as deep Gaussian process for the UQ and UP of multi-scale models. Deep GP is an efficient approach for deep learning and modelling of high-dimensional complex systems and consists of several hidden layers connected with non-linear mappings. In this work, a modified variational approximation framework is presented to analytically approximate the posterior distribution of the model outputs using the model inputs. The proposed framework automatically selects the dimensionality of each hidden layer to prevent over-fitting. Consequently, the deep GP model is more flexible and can be used to propagate uncertainty obtained in each layer across the hierarchy. The suitability of this model is shown in UQ tasks by computing the error bars for the statistics of interest in a standard stochastic PDE based model.

show abstract

Section: Learning Of the Bayesian Gaussian Process Latent Variable Mo...mentioning

confidence: 99%

Uncertainty Quantification of PDE-based models using Deep Gaussian Processes and Variational Bayesian Inference

Fanous,

Chatrabgoun,

Esmaeilbeigi

et al. 2023

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The methods are called surrogate modeling in cases when the original computational model represents a performance function. Popular surrogate modelling techniques include Gaussian process regression (Kriging surrogate model) [1][2], polynomial chaos expansions (PCE) [3][4][5], low-rank tensor approximations (LRA) [6] and support vector regression (SVR) [7][8]. A tricky problem that needs to be solved for surrogate-assisted UQ methods is that the required computational effort grows fast as the dimension of the input space increases.…”

Section: Introductionmentioning

confidence: 99%

An active-subspace-enhanced support vector regression model for high-dimensional uncertainty quantification

Zhou,

Gong,

Zhang

2024

Preprint

View full text Add to dashboard Cite

The computational costs of surrogate model-assisted uncertainty quantification methods become intractable for high dimensional problems. However, many high-dimensional problems are intrinsically low dimensional, if the output response exhibits some special structure that can be exploited within a low-dimensional subspace, known as the active subspace in the literature. Active subspace extracts linear combinations of all the original inputs, which may obscure the fact that only several inputs are active in the low-dimensional space. Motivated by this fact, this paper proposes a new surrogate modeling method which imposes sparsity in the active subspace to achieve a better performance for dimension reduction. Information given by sparse active subspace is integrated in the kernel structure of the support vector regression model to ensure superior performance for high dimensional problems. We demonstrate the proposed method on several benchmark applications, comprising an analytical function and two engineering applications of increasing dimensionality and complexity.

show abstract

Hybrid kernel approach to improving the numerical stability of machine learning for parametric equations with Gaussian processes in the noisy and noise-free data assumptions

Esmaeilbeigi

Cheraghi

2023

Engineering with Computers

View full text Add to dashboard Cite

On the impact of prior distributions on efficiency of sparse Gaussian process regression

Abstract: Author post-print (accepted) deposited by Coventry University's Repository Original citation & hyperlink: Esmaeilbeigi, M., Chatrabgoun, O., Daneshkhah, A. et al. On the impact of prior distributions on efficiency of sparse Gaussian process regression. Engineering with Computers (2022).

Cited by 4 publications

References 31 publications

Uncertainty Quantification of PDE-based models using Deep Gaussian Processes and Variational Bayesian Inference

Uncertainty Quantification of PDE-based models using Deep Gaussian Processes and Variational Bayesian Inference

An active-subspace-enhanced support vector regression model for high-dimensional uncertainty quantification

Hybrid kernel approach to improving the numerical stability of machine learning for parametric equations with Gaussian processes in the noisy and noise-free data assumptions

Contact Info

Product

Resources

About