Improved Most Likely Heteroscedastic Gaussian Process Regression via Bayesian Residual Moment Estimator

Abstract

This paper proposes an improved most likely heteroscedastic Gaussian process (MLHGP) algorithm to handle a kind of nonlinear regression problems involving input-dependent noise. The improved MLHGP follows the same learning scheme as the current algorithm by use of two Gaussian processes (GPs), with the first GP for recovering the unknown function and the second GP for modeling the input-dependent noise. Unlike the current MLHGP pursuing an empirical estimate of the noise level which is provably biased in most of local noise cases, the improved algorithm gives rise to an approximately unbiased estimate of the input-dependent noise. The approximately unbiased noise estimate is elicited from Bayesian residuals by the method of moments. As a by-product of this improvement, the expectation maximization (EM)-like procedure in the current MLHGP is avoided such that the improved algorithm requires only standard GP learnings to be performed twice. Four benchmark experiments, consisting of two synthetic cases and two real-world datasets, demonstrate that the improved MLHGP algorithm outperforms the current version not only in accuracy and stability, but also in computational efficiency. Index Terms-Gaussian process regression, most likely heteroscedastic Gaussian process, input-dependent noise, Bayesian residual, method of moments. Qiu-Hu Zhang received his B.Eng. and M.Sc. degrees from Hefei University of Technology, China, in 2009 and 2014, respectively. He is currently pursuing Ph.D. degree at The Hong Kong Polytechnic University, Hong Kong. His research interests include Bayesian machine learning, Bayesian decision theory, and structural health monitoring. Yi-Qing Ni received his B.Eng. degree

Static Attitude Determination Using Convolutional Neural Networks

“…Notice that the regular dropout could be interpreted as a Bayesian approximation of a Gaussian process [ 31 ]. By applying the dropout during both training and inference, it is possible to analyze it as if predictions from many different networks have been made, that is, a Monte Carlo sample from the space of all possible networks.…”

Stochastic emulation with enhanced partial‐ and no‐replication strategies for seismic response distribution estimation

“…This sample deviation can be used as an alternative to Equation () for the variance‐field observations, and requires to first establish the prediction of $$\hat{m}({{\bf x}})$$. Different smoothing techniques 44–46 can be used to approximate $$\hat{m}({{\bf x}})$$, and the one adopted here is to introduce an additional GP with homoskedesity assumption 30,36 . This tertiary GP will be denoted herein as GP m .…”

“…Different smoothing techniques [44][45][46] can be used to approximate m(𝐱), and the one adopted here is to introduce an additional GP with homoskedesity assumption. 30,36 This tertiary GP will be denoted herein as GP m . The workflow for this updated stochastic emulation algorithm, denoted ER-SE, is shown in Figure 1 and summarized below.…”

“…In particular, it enables the use of non-replicated samples in the variance estimation by adopting the techniques discussed in refs. [30,36] to replace the observations of sample variance with the squared sample deviation from the mean. To estimate this deviation a primitive mean estimate at each sample location is required, obtained by introducing an additional, tertiary, GP model relying on a homoscedasticity assumption.…”

A novel method for estimating utility harmonic impedance based probabilistic evaluation

“…The most commonly used method for probabilistic evaluation is the Bayesian theory, which utilises sampling information to correct prior probabilities to obtain posterior probabilities so that the distribution of posterior probabilities moves closer to the true distribution [18–21]. The Bayes formula [20] is $$\begin{equation}p(a\left| b \right.)$$…”