A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks

Abstract

In this paper, we develop an online sequential learning algorithm for single hidden layer feedforward networks (SLFNs) with additive or radial basis function (RBF) hidden nodes in a unified framework. The algorithm is referred to as online sequential extreme learning machine (OS-ELM) and can learn data one-by-one or chunk-by-chunk (a block of data) with fixed or varying chunk size. The activation functions for additive nodes in OS-ELM can be any bounded nonconstant piecewise continuous functions and the activation functions for RBF nodes can be any integrable piecewise continuous functions. In OS-ELM, the parameters of hidden nodes (the input weights and biases of additive nodes or the centers and impact factors of RBF nodes) are randomly selected and the output weights are analytically determined based on the sequentially arriving data. The algorithm uses the ideas of ELM of Huang et al. developed for batch learning which has been shown to be extremely fast with generalization performance better than other batch training methods. Apart from selecting the number of hidden nodes, no other control parameters have to be manually chosen. Detailed performance comparison of OS-ELM is done with other popular sequential learning algorithms on benchmark problems drawn from the regression, classification and time series prediction areas. The results show that the OS-ELM is faster than the other sequential algorithms and produces better generalization performance.

Fuzzy Multi-Objective Optimization Algorithms for Solving Multi-Mode Automated Guided Vehicles by Considering Machine Break Time and Artificial Neural Network

“…In order to solve this problem, OS-ELM algorithm uses the recursive idea to update the output weights online with new samples. Through some Benchmark data sets, the author pointed out that OS-ELM has achieved excellent performance in classification and regression [26]. At the same time, many scholars have also improved the OS-ELM algorithm.…”

“…The proposed adaptive OS-ELM based on improved ABC algorithm is compared with OS-ELM [26], EI-ELM [19], EM-ELM [11], and standard ELM [17]. The parameters of the proposed adaptive OS-ELM are as the follows: the data embedding dimension m is determined as 48, the number of neurons in the hidden It can be observed from Tab.…”

“…Based on standard ELM algorithm, the authors proposed an online sequential extreme learning machine (OS-ELM) algorithm [26]. OS-ELM first calculates its initial network weights in the initial training stage, and then the corresponding network weights can be obtained on the basis of the initial network weights when a new training sample is added to the training sample set.…”

Short-term Wind Power Prediction Using GA-ELM

“…Extreme learning machine [14,15], proposed by , is a sort of single-hidden layer feedforward neural networks (SLFNs). In ELM, the SLFN weights and biases are randomly initialized, and the output weight is determined then.…”

A new linear parametrization for peak friction coefficient estimation in real time

“…One of the major disadvantages in this type of models is the high processing power required by conventional learning techniques, typically based on back-propagation and clearly inappropriate for real time implementation. To overcome this difficulty, we applied the Extreme Learning Machine (ELM) method [9], [10], which allowed us to transform the non-linear NN model in a linear parametrization, with a reduced complexity, easier and faster to train. Compared with other linear parameterizations presented in the literature [4]- [7], the method proposed in this article is shown to offer better estimation performance with less number of parameters, thus easier to identify in real-time.…”