A Hybrid Differential Evolution and Back-Propagation Algorithm for Feedforward Neural Network Training

Abstract

In this study a hybrid differential evolution-back-propagation algorithm to optimize the weights of feedforward neural network is proposed.The hybrid algorithm can achieve faster convergence speed with higher accuracy. The proposed hybrid algorithm combining differential evolution (DE) and back-propagation (BP) algorithm is referred to as DE-BP algorithm to train the weights of the feed-forward neural (FNN) network by exploiting global searching feature of the DE evolutionary algorithm and strong local searching ability of the BP algorithm. The DE has faster exploration property during initial stage of global search for the expense of convergence speed. On the contrary, the problem of random initialization of weights may lead to getting stuck at local minima of the gradient based BP algorithm. In the proposed hybrid algorithm, initially we use global searching ability of the DE to move towards global optimal solution in the search space for few generations by selecting good starting weights and then precise local gradient searching of the BP in that region to converge to the optimal solution with increased speed of convergence. The performance of proposed DE-BP is investigated on a couple of public domain datasets, the experimental results are compared with the BP algorithm, the DE evolutionary training algorithm and a hybrid real-coded GA with back-propagation (GA-BP) algorithm . The results show that the proposed hybrid DE-BP algorithm produce promising results in comparison with other training algorithms.

An Optimization Model for Sustainability Program

“…This type of reasoning is not suitable in a logical context but can be observed in human behavior (Ding et al 2014). The proposed back propagation algorithm is used to train the weights of the feed-forward neural (FNN) network as the back propagation algorithm is strong towards local searching ability (Sarangi et al 2013). The Sect.…”

A neural differential evolution identification approach to nonlinear systems and modelling of shape memory alloy actuator

“…Many researchers have recognized that the combination of global search methods such as GA, PSO, and DE with local search methods like BP, Levenberg‐Marquardt (LM), and so on are effective approaches for building approximation models of ANN. Some studies hybridize the DE with gradient‐based methods . Following this trend, this paper presents a new way of integration of the BP into the MDE, in which, the BP is applied after the selection phase through the searching process of the MDE, and it only uses the best individual as an initial point for searching.…”

“…Curteanu et al proposed a new evolutionary neural method that applied a variant of DE using an opposition‐based learning initialization, a simple self‐adaptive procedure for the control parameters, and a modified mutation principle based on the fitness function as a criterion for reorganization. Authors in proposed the combination of a global search of the DE algorithm with local search methods like BP, Levenberg Marquardt (LM) to build an effective approximate approach for the neural network models.…”

Coastal Wetland Mapping with Sentinel-2 MSI Imagery Based on Gravitational Optimized Multilayer Perceptron and Morphological Attribute Profiles

“…This approach, on the other hand, easily falls into local optima and gets premature convergence [38,47]. In recent years, because of their promising self-organization and global optimization abilities, swarm intelligence algorithms, such as genetic algorithms (GAs) [40,48], particle swarm optimization (PSO) algorithms [46,49,50], and differential evolution (DE) [51] were successfully used to optimize the parameters of MLP.…”