Adapting modularity during learning in cooperative co-evolutionary recurrent neural networks

Abstract

Adaptation during evolution has been an important focus of research in training neural networks. Cooperative coevolution has played a significant role in improving standard evolution of neural networks by organizing the training problem into modules and independently solving them. The number of modules required to represent a neural network is critical to the success of evolution. This paper proposes a framework for the adaptation of the number of modules during evolution. The framework is called adaptive modularity cooperative coevolution. It is used for training recurrent neural networks on grammatical inference problems. The results shows that the proposed approach performs better than its counterparts as the dimensionality of the problem increases.

Competition and Collaboration in Cooperative Coevolution of Elman Recurrent Neural Networks for Time-Series Prediction

“…Competition can ensure that the different problem decomposition methods are given an opportunity during the entire evolution as opposed to adaptive problem decomposition method as in our previous work [16], [17] where the problem decomposition method is adapted over time. In this way, there is no problem in finding the right problem decomposition method at a particular time according to the degree of separability [4].…”

“…Adaptation of problem decomposition method at different stages of evolution is costly as it is difficult to establish optimal parameters that indicate when to switch from one problem decomposition to another and how long to use them [16]. Extensive experiments are needed when adaptation of problem decomposition is applied to different neural network architectures and problems [16].…”

“…Adaptation of problem decomposition in different phases of evolution has been effective for training feedforward networks on pattern recognition problems [15] and recurrent networks on grammatical inference problems [16]. Adaptation of problem decomposition method at different stages of evolution is costly as it is difficult to establish optimal parameters that indicate when to switch from one problem decomposition to another and how long to use them [16].…”

Cooperative neuro-evolution of Elman recurrent networks for tropical cyclone wind-intensity prediction in the South Pacific region

“…In addition to FMS, the Australian Bureau of Meteorology (BoM), a Tropical Cyclone Warning Center (TCWC), is also responsible for the far southwest Pacific Ocean basin 4 . While the U.S naval Joint Typhoon Warning Centre (JTWC) 5 is not a WMO recognised RSMC or TCWC, it however also issues cyclone warnings for various ocean basins, including northwest Pacific, North Indian, Southwest Indian, Southeast Indian/Australian, and the Australian/Southwest Pacific basins [Central Pacific Hurricane Centre] 6 .…”

“…CC has been effective for neuro-evolution of feedforward and recurrent neural networks [2], [3], [4], [5], [6], [7]. Problem decomposition is an important procedure in cooperation coevolution that determines how the subcomponents are decomposed which actually means dividing the neural network into smaller regions that are training cooperatively and collectively [2].…”