Pruning with Replacement on Limited Resource Allocating Networks by F-Projections

Abstract

The principle of F-projection, in sequential function estimation, provides a theoretical foundation for a class of gaussian radial basis function networks known as the resource allocating networks (RAN). The ad hoc rules for adaptively changing the size of RAN architectures can be justified from a geometric growth criterion defined in the function space. In this paper, we show that the same arguments can be used to arrive at a pruning with replacement rule for RAN architectures with a limited number of units. We illustrate the algorithm on the laser time series prediction problem of the Santa Fe competition and show that results similar to those of the winners of the competition can be obtained with pruning and replacement.

Online Modeling With Tunable RBF Network

“…Both RAN and RAN-EKF algorithms only grow the network size without a pruning strategy to remove the obsolete RBF nodes, so the model size can be too large in some applications. Hence in [16], an improved approach of the RAN algorithm was proposed by limiting the size of the RBF network (L-RAN). Further in [17], [18], a more compact model can be achieved by using the minimal RAN (M-RAN) algorithm which prunes the inactive kernel nodes based on relative contribution.…”

Sequential Hierarchical Pattern Clustering

“…Our approach to such very large scale clustering algorithms is to study sequential and constructive algorithms, much in the spirit of the resource allocating network by Platt [14] and its variants by Kadirkamanathan et al [7], and Molina et al [11]. Another online approach for large scale learning (> 4M datapoints) was proposed by Farran et al [2].…”

Sequential Learning for Adaptive Critic Design: An Industrial Control Application

“…The Critic Network is trained by using , the previous estimate of the cost, and the current reward, r(k), to provide a target value for the current cost estimate . Thus, (4) The instantaneous error, , is then a function of two successive values of : ,…”

“…The seminal paper by Platt [2], proved that sequential learning using a constructive technique, called resource allocation networks (RAN) is suitable for online modelling. Since then there have been many publications on research into application of this concept to supervised learning problems [3,4,5].…”