An online learning algorithm with dimension selection using minimal hyper basis function networks

Abstract

SUMMARYIn this study, the authors propose an HMRAN (Hyper MRAN) for which dimension selection is possible, by extending a minimal resource allocating network (MRAN) for online learning with minimized resources, with the application of a growth strategy and pruning strategy for hidden units in a Gaussian radial basis function (GRBF) network. When the input is a multidimensional pattern, it is often the case that dimensions with an explicit relationship to the output are limited to a single section and it is difficult to develop a correspondence with a MRAN having an isotropic Gaussian function as the basis function. Dimension selection can be flexibly performed in basis function units with the use of a hyper basis function (Hyper BF) instead of a Gaussian function. Furthermore, computational complexity is reduced by using a localized extended Kalman filter (LEKF) and the introduction of a merging strategy for the basis function and improvements to the pruning strategy resulted in a reduction of the resources needed. A number of benchmark tests were performed and it was confirmed that the proposed technique can be implemented with good accuracy and minimal resources compared to the conventional technique.

An Overview of Sequential Learning Algorithms for Single Hidden Layer Networks: Current Issues & Future Trends

“…To overcome this problem with MRAN, extended minimal resource network (EMRAN) was proposed in [40] but this algorithm lags reasonable accuracy. In [16], Hyper MRAN (HMRAN) was formulated, with suitable input data sample dimension selection with hyper radial basis function (Hyper RBF) as an activation function for SHLFNs, to reduce the time complexity problem of MRAN by using localized extended Kalman filter approach also the hyper RBF activation function ensure high accuracy.…”

“…The performance of this SHLFNs onlinesequential learning algorithm was improved over the MRAN algorithm. The parameters in MRAN are adjusted at each iteration using the extended Kalman filter method, which leads to the increment in the hidden layer neurons during the parameter update process [16]. Also, the size of the covariance matrix used in EKF is usually very large, which increases the computational complexity of the network structure.…”

“…To minimize the computational cost of the RANEKF learning algorithm, the MRAN algorithm added the deletion mechanism for the neurons in the hidden layer [15]. An improved MRAN learning algorithm (Hyper-MRAN) was proposed in 2006 which uses a new weight adjustment strategy that reduces the disadvantages of high memory requirements of the MRAN algorithm with higher accuracy [16].…”

Dynamic Threshold Selection for Sequential Learning in Radial Basis Function Networks

“…Ever since the pruning algorithm in MRAN was introduced, the MRAN algorithm has been successfully applied and furthermore, improvised to attain optimal neuron configurations in the hidden layer as in EMRAN and HMRAN developed by Li et al and Nishida et al in [9], [10] respectively. By removing unnecessary neurons in the hidden layer of the network, minimal computation power is required in the actual implementation of the system.…”