Evolving Deep Neural Networks via Cooperative Coevolution With Backpropagation

Gong, Maoguo; Liu, Jia; Qin, A. K.; Zhao, Kun; Tan, Kay Chen

doi:10.1109/tnnls.2020.2978857

Cited by 25 publications

(13 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…According to Lemma 11, V 1 (t) = 0 when t > t * , which illustrates that network (2) achieves FTS under controller (5). ▪ Now, we discuss a special case of network (2), that is, when the feedback gain 𝜀 1 = 0, the state feedback controller ( 5) is reduced to…”

Section: Fixed Coupling Weightsmentioning

confidence: 99%

“…In recent years, neural networks have received a lot of attention because of their extensive application in some fields, and a number of results have been reported in the literature, such as convolutional neural networks 1 and backpropagation neural networks. 2 Coupled neural networks (CNNs) as a special case of neural networks are made up of multiple interconnected neurons of neural networks, which can be applied to model a wide range of actual networks, including biological networks 3 and Internet. 4 Moreover, CNNs have a more sophisticated dynamic feature in contrast to regular neural networks.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Finite‐time synchronization of multi‐weighted fractional‐order coupled neural networks with fixed and adaptive couplings

Tong

Chen

et al. 2022

Adaptive Control & Signal

View full text Add to dashboard Cite

This article concentrates on the finite-time synchronization (FTS) for multi-weighted fractional-order coupled neural networks (MFCNNs) with fixed and adaptive couplings. A new dynamic model, which involves coupled neural networks with fractional order and multi-weighted couplings, is proposed. Furthermore, an adaptive law to adjust the coupling weights is devised to ensure the FTS of such a complex model. First, by using fractional-order calculus properties and inequality techniques, a finite-time fractional differential inequality is presented, which considerably extends the prior result. Second, by the Lyapunov stability theory and the fractional-order inequality, several sufficient conditions are acquired to make sure the FTS for MFCNNs. Moreover, the influence of the system's order on the FTS is also uncovered. Finally, a numerical simulation example and a practical simulation example are provided to demonstrate the validity of the obtained criteria.

show abstract

Section: Fixed Coupling Weightsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Finite‐time synchronization of multi‐weighted fractional‐order coupled neural networks with fixed and adaptive couplings

Tong

Chen

et al. 2022

Adaptive Control & Signal

View full text Add to dashboard Cite

show abstract

“…3) The final is using the EA and the gradient-based method alternatively where the output of one method servers the start point for another, and this procedure is iterated until some stopping criteria are met. Existing works mainly differ in the order of applying these two methods in the iterative procedure and the individuals chosen from being continually optimized by using the gradient-based method [38], [41]- [43], [47]. For example, in [42], the EA is applied first, and the top 10% individuals with higher fitness in the final population of the EA are further optimized by the gradient-based methods.…”

Section: B Taxonomy and Survey Of Existing Ea-based Approaches 1) Taxonomymentioning

confidence: 99%

“…For example, in [42], the EA is applied first, and the top 10% individuals with higher fitness in the final population of the EA are further optimized by the gradient-based methods. In [38], the model is trained by using the gradient-based method until the performance improvement is below a certain threshold. Then, the EA is applied to further optimize model parameters.…”

Section: B Taxonomy and Survey Of Existing Ea-based Approaches 1) Taxonomymentioning

confidence: 99%

A Survey on Evolutionary Construction of Deep Neural Networks

Xun

Qin

Gong

et al. 2021

IEEE Trans. Evol. Computat.

Self Cite

View full text Add to dashboard Cite

Automated construction of deep neural networks (DNNs) has become a research hot spot nowadays because DNN's performance is heavily influenced by its architecture and parameters, which are highly task-dependent, but it is notoriously difficult to find the most appropriate DNN in terms of architecture and parameters to best solve a given task. In this work, we provide an insight into the automated DNN construction process by formulating it into a multilevel multiobjective large-scale optimization problem with constraints, where the nonconvex, nondifferentiable, and black-box nature of this problem make evolutionary algorithms (EAs) to stand out as a promising solver. Then, we give a systematical review of existing evolutionary DNN construction techniques from different aspects of this optimization problem and analyze the pros and cons of using EA-based methods in each aspect. This work aims to help DNN researchers to better understand why, where, and how to utilize EAs for automated DNN construction and meanwhile, help EA researchers to better understand the task of automated DNN construction so that they may focus more on EA-favored optimization scenarios to devise more effective techniques.

show abstract

“…Likewise, Mandal et al [12] also used BP ANN in simulating As(III) removal with R 2 above 0.97 for both training and validation processes. However, the drawbacks of BP ANN that it tends to be trapped by local optima due to severe initialization sensitivity are often put forward by researchers [13]. Apart from that, the high requirements for computational complexity and memory in some BP ANN intrinsic algorithms like Levenberg-Marquardt also deserve proper attention [14].…”

Section: Application Of the Ann Model On Modelling The Watermentioning

confidence: 99%

Employing Artificial Neural Networks to Predict the Performance of Domestic Sewage Treatment Terminals in the Rural Region

Lin

Luo

Zhang

et al. 2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Domestic sewage in rural regions is mainly treated by small-scale treatment terminals in China. The large quantities and high dispersion of these terminals render the chemical measurement of effluent to be a time and energy intensive work and further hinder the efficient surveillance of terminals’ performance. After a thorough investigation of 136 operating terminals, this study successfully employs two artificial neural network (ANN) models to predict effluent total nitrogen (TN) and COD (R2 both higher than 0.8) by setting some easily detectable parameters, e.g., pH and conductivity, as inputs. To prevent ANN models getting stuck on local optima and enhance the model performance, genetic algorithm (GA) and particle swarm optimization (PSO) are introduced into ANN, respectively. By comparison, ANN-PSO excels in modelling both TN and COD. The root mean square error (RMSE) and R2 of ANN-PSO in modelling TN are 9.14 and 0.90, respectively, in the training stage, and 11.54 and 0.90, respectively, in the validation stage. The RMSE and R2 of ANN-PSO in modelling COD are 22.10 and 0.90, respectively, in the training stage, and 26.57 and 0.85, respectively, in the validation stage. This is the first study to provide performance prediction models that are available for different terminals. Two established ANN-PSO models show great practical significance in monitoring huge amounts of terminals despite the slight sacrifice of models’ accuracy caused by the great heterogeneity of different terminals.

show abstract

Evolving Deep Neural Networks via Cooperative Coevolution With Backpropagation

Cited by 25 publications

References 33 publications

Finite‐time synchronization of multi‐weighted fractional‐order coupled neural networks with fixed and adaptive couplings

Finite‐time synchronization of multi‐weighted fractional‐order coupled neural networks with fixed and adaptive couplings

A Survey on Evolutionary Construction of Deep Neural Networks

Employing Artificial Neural Networks to Predict the Performance of Domestic Sewage Treatment Terminals in the Rural Region

Contact Info

Product

Resources

About