A review of Bayesian neural networks with an application to near infrared spectroscopy

Abstract

MacKay's (1992) Bayesian framework for backpropagation is a practical and powerful means to improve the generalization ability of neural networks. It is based on a Gaussian approximation to the posterior weight distribution. The framework is extended, reviewed, and demonstrated in a pedagogical way. The notation is simplified using the ordinary weight decay parameter, and a detailed and explicit procedure for adjusting several weight decay parameters is given. Bayesian backprop is applied in the prediction of fat content in minced meat from near infrared spectra. It outperforms "early stopping" as well as quadratic regression. The evidence of a committee of differently trained networks is computed, and the corresponding improved generalization is verified. The error bars on the predictions of the fat content are computed. There are three contributors: The random noise, the uncertainty in the weights, and the deviation among the committee members. The Bayesian framework is compared to Moody's GPE (1992). Finally, MacKay and Neal's automatic relevance determination, in which the weight decay parameters depend on the input number, is applied to the data with improved results.

Online Learning Solutions for Freeway Travel Time Prediction

“…, N k . The benefit of translating cost function C (3) into C * (6) is that Bayes' rule (used to calculate the posterior) automatically embodies Occam's razor [19], and that minimizing C * , thus, leads to the simplest setting of ψ that is still warranted by the data D. In [19] and [24], it is shown that the maximum likelihood estimates for α and β can be expressed as follows:…”

“…At each time period p > k for which no realized travel time d k of vehicles departing at k is available, the censored error (24) provides an incremental estimate of the model prediction error. Letting…”

Bagging for robust non-linear multivariate calibration of spectroscopy

“…Bayesian back-propagation was initially introduced by MacKay [47][48], based on a Gaussian approximation to the posterior distribution of parameters. Previous study has shown that the Bayesian approach to parameter estimation typically attains more robust and accurate ANN models [49].…”

Bayesian variable selection for Gaussian process regression: Application to chemometric calibration of spectrometers

“…The wide acceptance of these linear methods is largely due to the inherent linearity between the analyte properties and spectral absorbance as stated in the Beer-Lambert's law [4]. However non-linear calibration methods have also been proposed for practical problems where the linearity does not hold, including neural networks [5], support vector machine and its variants [6,7].…”