Hybrid extreme learning machine approach for heterogeneous neural networks

Abstract

In this paper, a hybrid learning approach, which combines the extreme learning machine (ELM) with a genetic algorithm (GA), is proposed. The utilization of this hybrid algorithm enables the creation of heterogeneous single layer neural networks (SLNNs) with better generalization ability than traditional ELM in terms of lower mean square error (MSE) for regression problems or higher accuracy for classification problems. The architecture of this method is not limited to traditional linear neurons, where each input participates equally to the neuron's activation, but is extended to support higher order neurons which affect the network's generalization ability. Initially, the proposed heterogeneous hybrid extreme learning machine (He-HyELM) algorithm creates a number of custom created neurons with different structure, which are used for the creation of homogeneous SLNNs. These networks are trained with ELM and an application specific GA evolves them into heterogeneous networks according to a fitness criterion utilizing the uniform crossover operator for the recombination process. After the completion of the evolution process, the network with the best fitness is selected as the most optimal.

A Systematic Error Compensation Method Based on an Optimized Extreme Learning Machine for Star Sensor Image Centroid Estimation

“…For example, the poor performance indices and low learning rate of the BP algorithm, along with how easily it becomes trapped in a local optimum, limit the compensation accuracy of this method; the simplified approximation and iteration estimation in the analytical compensation method lead to a reduction in the prediction accuracy and high on-line computational complexity, which is not suitable for the on-orbit embedded system; the bivariate polynomial compensation template is valid only for some specific Gaussian width cases, and its application range is thus limited; and the problem of scientifically setting the penalty factor and kernel parameter in LSSVR still remains unsettled, meaning that model training is more difficult because of a time-consuming parameter selection process. ELM [24,25] has attracted attention in robot control [26], human face recognition [27], medical diagnosis [28], sales forecasting [29], and protein structure prediction [30] fields, among others, due to its simple training process and excellent generalization ability when compared to other traditional algorithms [31]. However, the randomness of input weights and hidden layer biases become a bottleneck restricting the stability and prediction accuracy of the ELM network.…”

A Novel Data Fusion Strategy Based on Extreme Learning Machine Optimized by Bat Algorithm for Mobile Heterogeneous Wireless Sensor Networks

“…The variable b j is the hidden layer threshold; β j is the connection weight between the hidden layer and the output layer. The variable O j is the output of the network [25].…”

High-precision calculation of gas saturation in organic shale pores using an intelligent fusion algorithm and a multi-mineral model

“…The ELM algorithm, which was initially proposed by Guangbin Huang in 2004 (Christou et al, 2018), is characterized by randomly or artificially assigned weights of the hidden layer nodes and very fast training and prediction phases. The hidden layer weights do not need to be updated, and the random weights and learning process calculate only the output weight.…”