Ensemble of heterogeneous flexible neural trees using multiobjective genetic programming

Abstract

Machine learning algorithms are inherently multiobjective in nature, where approximation error minimization and model's complexity simplification are two conflicting objectives. We proposed a multiobjective genetic programming (MOGP) for creating a heterogeneous flexible neural tree (HFNT), tree-like flexible feedforward neural network model. The functional heterogeneity in neural tree nodes was introduced to capture a better insight of data during learning because each input in a dataset possess different features.MOGP guided an initial HFNT population towards Pareto-optimal solutions, where the final population was used for making an ensemble system. A diversity index measure along with approximation error and complexity was introduced to maintain diversity among the candidates in the population. Hence, the ensemble was created by using accurate, structurally simple, and diverse candidates from MOGP final population. Differential evolution algorithm was applied to fine-tune the underlying parameters of the selected candidates. A comprehensive test over classification, regression, and time-series datasets proved the efficiency of the proposed algorithm over other available prediction methods. Moreover, the heterogeneous creation of HFNT proved to be efficient in making ensemble system from the final population.

Neural Tree for Estimating the Uniaxial Compressive Strength of Rock Materials

“…The internal node V is a set of internal (computational) nodes and, the leaf node T is a set of inputs [17]. Hence, an HFNT M G can be expressed as:…”

“…The developed best HFNT M model using the parameter setting mentioned in [17] and two-fold training is shown in Fig. 1, where the leaf nodes indicate the input features and the root node gives the predicted output UCS of the model.…”

“…Interestingly, HFNT M differs from NN [18] in its structural configuration, and it differs from the commonly used regression tree [19] in its node type. Moreover, the tree-like structure in the HFNT M is created by using multiobjective genetic programming (MOGP) [17,20]. Therefore, the primary advantages of using HFNT M over other CI techniques lie in its ability of the automatic adaptation into the structure and the input feature selection.…”

“…The multiobjective heterogeneous flexible neural tree (HFNT M ) produces a tree-like model [17], in which the nodes of the tree are similar to the NN nodes was used in this study. Interestingly, HFNT M differs from NN [18] in its structural configuration, and it differs from the commonly used regression tree [19] in its node type.…”

Metaheuristic design of feedforward neural networks: A review of two decades of research

“…Infact a paradigm, called Neuroevolution that accommodates adaptive learning all or some components of FNN in some intuitive ways by applying EAs. For examples, generalized acquisition of recurrent links (GNARL) [257], evolutionary programming net (EPNet) [258], neuroevolution of augmenting topologies (NEAT) [259], hypercube-based neuroevolution of augmenting topologies (Hyper-NEAT) [260], evolutionary acquisition of neural topologies (EANT2) [261], and heterogeneous flexible neural tree (HFNT) [262] optimizes both FNN structure and parameters (weights) using some direct or indirect encoding methods. Moreover, several other paradigms and methods proposed in the past for the simultaneous optimization of FNN components are described as follows.…”