Online Learning with Radial Basis Function Networks

Abstract

We investigate the benefits of feature selection, nonlinear modelling and online learning when forecasting financial time series. We combine sequential updating with continual learning, specifically transfer learning. We perform feature representation transfer through clustering algorithms that determine the analytical structure of radial basis function networks we construct. These networks achieve lower mean-square prediction errors than kernel ridge regression models, which arbitrarily use all training vectors as prototypes. We also demonstrate quantitative procedures to determine the very structure of the networks. Finally, we conduct experiments on the log-returns of financial time series and show that these online transfer learning models outperform a random-walk baseline. In contrast, the offline learning models struggle to do so.

Sequential asset ranking in nonstationary time series

“…It is first with Moody and Darken [24] that we see the formulation of the rbfnet as the combination of an unsupervised learning model (k-means) and a supervised learning model (linear regression). This form of feature representation transfer boosts model learning capacity and, when combined with sequential optimisation, outperforms biased baselines such as the random-walk in multi-horizon returns forecasting [4].…”

“…We use their caw procedure but combine it with exponentially weighted recursive least-squares to facilitate sequential optimisation in the test set without forward-looking bias. Finally, we utilise the research of Borrageiro et al [4] to make use of online learning rbfnets, as these models retain more remarkable knowledge of the input feature space. They also respond better to regime changes or concept drifts than models that do not use feature representation transfer; for example, from clustering algorithms [4], Gaussian mixture models [6] or echo state networks [5].…”