Efficient reinforcement learning through symbiotic evolution

Abstract

Abstract.This article presents a new reinforcement learning method called SANE (Symbiotic, Adaptive Neuro-Evolution), which evolves a population of neurons through genetic algorithms to form a neural network capable of performing a task. Symbiotic evolution promotes both cooperation and specialization, which results in a fast, efficient genetic search and discourages convergence to suboptimal solutions. In the inverted pendulum problem, SANE formed effective networks 9 to 16 times faster than the Adaptive Heuristic Critic and 2 times faster than Q-learning and the GENITOR neuro-evolution approach without loss of generalization. Such efficient learning, combined with few domain assumptions, make SANE a promising approach to a broad range of reinforcement learning problems, including many real-world applications.

An Improved Genetic Algorithm

“…Cooperative Evolutionary Algorithm [8] is developed to realize what has just been mentioned above. Potter proposed a model about Cooperative Evolutionary Algorithm through lots of researching; Potter and De Jong [9] solved the extreme value of the function of several variables; Moriarity [10] used Cooperative Evolutionary Algorithm for the evolution of three-layered feedforward neural network, and proposed Symbiotic Adaptive Neuro-Evolution Algorithm.…”

Two-Strategy reinforcement group cooperation based symbiotic evolution for TSK-type fuzzy controller design

“…Recently, many researches try to enhance the traditional GAs have been made [29,[30][31][32] . One category of them tries to modify the structure of a population.…”

“…One category of them tries to modify the structure of a population. Examples in this category include the distributed GA [31] , the cellular GA [31] , and the symbiotic GA [32] .…”

“…This study proposes the group cooperation based symbiotic evolution (GCSE) to improve the symbiotic GA [31] . In the proposed GCSE, the algorithm is developed from symbiotic evolution.…”

“…This problem is often used as an example of inherently unstable and dynamic systems to demonstrate both modern and the classic control techniques [30,31,32] , or the reinforcement learning schemes [28] , and is now used as a control benchmark. The cart-pole balancing problem is the problem of learning how to balance an upright pole.…”