Fast methods for training Gaussian processes on large datasets

Moore, C. J.; Chua, Alvin J. K.; Berry, C. P. L.; Gair, J. R.

doi:10.1098/rsos.160125

Cited by 39 publications

(20 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If its gradient is known and a gradient-based algorithm, such as a conjugate gradient method, can be used (as proposed in [45]), the maximisation process may be accelerated. More information about using the gradient and Hessian of Equation (21) to speed the learning phase in the GPR algorithm can be found in [28,46,47].…”

Section: Training a Gpr Modelmentioning

confidence: 99%

Multi-Horizon Forecasting of Global Horizontal Irradiance Using Online Gaussian Process Regression: A Kernel Study

et al. 2020

View full text Add to dashboard Cite

In the present paper, global horizontal irradiance (GHI) is modelled and forecasted at time horizons ranging from 30 min to 48 h, thus covering intrahour, intraday and intraweek cases, using online Gaussian process regression (OGPR) and online sparse Gaussian process regression (OSGPR). The covariance function, also known as the kernel, is a key element that deeply influences forecasting accuracy. As a consequence, a comparative study of OGPR and OSGPR models based on simple kernels or combined kernels defined as sums or products of simple kernels has been carried out. The classic persistence model is included in the comparative study. Thanks to two datasets composed of GHI measurements (45 days), we have been able to show that OGPR models based on quasiperiodic kernels outperform the persistence model as well as OGPR models based on simple kernels, including the squared exponential kernel, which is widely used for GHI forecasting. Indeed, although all OGPR models give good results when the forecast horizon is short-term, when the horizon increases, the superiority of quasiperiodic kernels becomes apparent. A simple online sparse GPR (OSGPR) approach has also been assessed. This approach gives less precise results than standard GPR, but the training computation time is decreased to a great extent. Even though the lack of data hinders the training process, the results still show the superiority of GPR models based on quasiperiodic kernels for GHI forecasting.

show abstract

Section: Training a Gpr Modelmentioning

confidence: 99%

Multi-Horizon Forecasting of Global Horizontal Irradiance Using Online Gaussian Process Regression: A Kernel Study

et al. 2020

View full text Add to dashboard Cite

show abstract

“…This is a major advantage for its use despite the limitations of training. Recent techniques to address the issue of training Gaussian process models for large datasets could be a way ahead in future studies [77] . Another option is to use Bayesian neural networks, rather than conventional neural networks for the choice of the surrogate model.…”

Section: Discussionmentioning

confidence: 99%

Langevin-gradient parallel tempering for Bayesian neural learning

et al. 2019

View full text Add to dashboard Cite

Parallel tempering addresses some of the drawbacks of canonical Markov Chain Monte-Carlo methods for Bayesian neural learning with the ability to utilize high performance computing. However, certain challenges remain given the large range of network parameters and big data. Surrogate-assisted optimization considers the estimation of an objective function for models given computational inefficiency or difficulty to obtain clear results. We address the inefficiency of parallel tempering for large-scale problems by combining parallel computing features with surrogate assisted estimation of likelihood function that describes the plausibility of a model parameter value, given specific observed data. In this paper, we present surrogateassisted parallel tempering for Bayesian neural learning where the surrogates are used to estimate the likelihood. The estimation via the surrogate becomes useful rather than evaluating computationally expensive models that feature large number of parameters and datasets. Our results demonstrate that the methodology significantly lowers the computational cost while maintaining quality in decision making using Bayesian neural learning. The method has applications for a Bayesian inversion and uncertainty quantification for a broad range of numerical models.

show abstract

“…The covariance function, and any free parameters therein, are free to be specified; however, they can also be learnt from the training set by maximizing the probability of the training set being realized by the GP (maximizing the GP evidence). This learning process can be computationally expensive, especially for large training sets or when comparing covariance functions with many free parameters; the techniques described in [100] were used to accelerate this learning phase. The covariance functions considered here were the squared-exponential and Wendland polynomial functions used previously for waveform modeling in [43]; these covariance functions are all stationary, i.e.…”

Section: B Merger and Ringdownmentioning

confidence: 99%

Eccentric, nonspinning, inspiral, Gaussian-process merger approximant for the detection and characterization of eccentric binary black hole mergers

et al. 2018

View full text Add to dashboard Cite

We present ENIGMA, a time domain, inspiral-merger-ringdown waveform model that describes non-spinning binary black holes systems that evolve on moderately eccentric orbits. The inspiral evolution is described using a consistent combination of post-Newtonian theory, self-force and black hole perturbation theory. Assuming eccentric binaries that circularize prior to coalescence, we smoothly match the eccentric inspiral with a stand-alone, quasi-circular merger, which is constructed using machine learning algorithms that are trained with quasi-circular numerical relativity waveforms. We show that ENIGMA reproduces with excellent accuracy the dynamics of quasi-circular compact binaries. We validate ENIGMA using a set of Einstein Toolkit eccentric numerical relativity waveforms, which describe eccentric binary black hole mergers with mass-ratios between 1 ≤ q ≤ 5.5, and eccentricities e0 ∼ < 0.2 ten orbits before merger. We use this model to explore in detail the physics that can be extracted with moderately eccentric, non-spinning binary black hole mergers. In particular, we use ENIGMA to show that the gravitational wave transients GW150914, GW151226, GW170104, GW170814 and GW170608 can be effectively recovered with spinning, quasi-circular templates if the eccentricity of these events at a gravitational wave frequency of 10Hz satisfies e0 ≤ {0.175, 0.125, 0.175, 0.175, 0.125}, respectively. We show that if these systems have eccentricities e0 ∼ 0.1 at a gravitational wave frequency of 10Hz, they can be misclassified as quasi-circular binaries due to parameter space degeneracies between eccentricity and spin corrections. Using our catalog of eccentric numerical relativity simulations, we discuss the importance of including higher-order waveform multipoles in gravitational wave searches of eccentric binary black hole mergers.

show abstract

Fast methods for training Gaussian processes on large datasets

Cited by 39 publications

References 26 publications

Multi-Horizon Forecasting of Global Horizontal Irradiance Using Online Gaussian Process Regression: A Kernel Study

Multi-Horizon Forecasting of Global Horizontal Irradiance Using Online Gaussian Process Regression: A Kernel Study

Langevin-gradient parallel tempering for Bayesian neural learning

Eccentric, nonspinning, inspiral, Gaussian-process merger approximant for the detection and characterization of eccentric binary black hole mergers

Contact Info

Product

Resources

About