Improving Classification Performance through an Advanced Ensemble Based Heterogeneous Extreme Learning Machines

Abstract

Extreme Learning Machine (ELM) is a fast-learning algorithm for a single-hidden layer feedforward neural network (SLFN). It often has good generalization performance. However, there are chances that it might overfit the training data due to having more hidden nodes than needed. To address the generalization performance, we use a heterogeneous ensemble approach. We propose an Advanced ELM Ensemble (AELME) for classification, which includes Regularized-ELM, L2-norm-optimized ELM (ELML2), and Kernel-ELM. The ensemble is constructed by training a randomly chosen ELM classifier on a subset of training data selected through random resampling. The proposed AELM-Ensemble is evolved by employing an objective function of increasing diversity and accuracy among the final ensemble. Finally, the class label of unseen data is predicted using majority vote approach. Splitting the training data into subsets and incorporation of heterogeneous ELM classifiers result in higher prediction accuracy, better generalization, and a lower number of base classifiers, as compared to other models (Adaboost, Bagging, Dynamic ELM ensemble, data splitting ELM ensemble, and ELM ensemble). The validity of AELME is confirmed through classification on several real-world benchmark datasets.

Fast Local Laplacian‐Based Steerable and Sobel Filters Integrated with Adaptive Boosting Classification Tree for Automatic Recognition of Asphalt Pavement Cracks

“…AdaBoost Advances in Civil Engineering ensemble of CTrees can be defined as a combination of multiple CTrees in which the final prediction result is obtained by combining the outputs of individual trees. Based on previous works [52][53][54][55][56], ensemble models have demonstrated better performance than individual models in a wide range of applications. e AdaBoost algorithm is demonstrated in Figure 6.…”

Voting-Boosting: A novel machine learning ensemble for the prediction of Infants' Data

“…Ensemble method is a machine learning technique having collection of several classifiers to improve the overall predictive performance (28,29) . The present section of model building & evaluation deals with heterogeneous ensemble called Voting-Boosting (VB) ensemble and its architecture is shown in Figure 2.…”

Hybrid extreme learning machine approach for heterogeneous neural networks

“…This approach has shown to be effective for both regression and classification problems although it may require more nodes than networks trained with gradient-based methods like back-propagation [24] due to the randomization of the hidden layer weights and thresholds [72,69]. Furthermore, the ELM algorithm is unable to create SLNNs containing different combinations of neuron types 1 in the hidden layer (these network types will 1 The term different combinations of neuron types refers to neuron types with structural units like the structured composite model (C-Model) proposed by Christou et al [9] which is utilized in the proposed algorithm. This model divides the neuron into three subunits (subcomponents) named dendrite (D), activation (S a ) function, activation-output (S ao ) function and is able to create a custom neuron by varying any of the subunits from each category.…”

“…The synaptic kernel inverse method (SKIM) by Tapson et al [59] for event-based systems redefines the hidden neurons as synaptic kernels in which the input event based signals are transformed into continuous-valued signals. The advanced ELM ensemble (AELME) from Abuassba et al [1] constructs a network ensemble by training a randomly chosen ELM classifier on a subset of training data selected through random resampling. These methodologies work only with original units and they are not taking into consideration higher order units.…”