Adaptive Genomic Evolution of Neural Network Topologies (AGENT) for State-to-Action Mapping in Autonomous Agents

Abstract

Neuroevolution is a process of training neural networks (NN) through an evolutionary algorithm, usually to serve as a state-to-action mapping model in control or reinforcement learning-type problems. This paper builds on the Neuro Evolution of Augmented Topologies (NEAT) formalism that allows designing topology and weight evolving NNs. Fundamental advancements are made to the neuroevolution process to address premature stagnation and convergence issues, central among which is the incorporation of automated mechanisms to control the population diversity and average fitness improvement within the neuroevolution process. Insights into the performance and efficiency of the new algorithm is obtained by evaluating it on three benchmark problems from the Open AI platform and an Unmanned Aerial Vehicle (UAV) collision avoidance problem.

Training Detection-Range-Frugal Cooperative Collision Avoidance Models for Quadcopters via Neuroevolution

“…Based on our observations in, 16 using smaller reaction time leads to a more controllable maneuver; these maneuvers are not necessarily energy optimal. Since the focus of the current paper is not about energy optimality but rather the detection quality which might be more useful in practice, it is possible to fix the reaction time to the smallest practical value.…”

“…22,23 Further description of our neuroevolution method called AGENT can be found in. 16 Neuroevolution uses an evolutionary algorithm to optimize the topology and weights of a neural network, and is typically used for solving problems that can be posed as reinforcement learning; 24 however unlike RL, neuroevolution is significantly more amenable to parallel deployment and escaping local minima, both crucial to expensive offline learning investments.…”

Anti-collision Technologies for Unmanned Aerial Vehicles: Recent Advances and Future Trends

“…[78] The adaptive differential multi-objective optimization algorithm was used to find the optimal solution to avoid obstacles. [79] Solved the problem of premature stagnation in genetic algorithm. [80] Applied genetic algorithm and evolutionary robot to evolve neural network controller.…”

“…[78] defined the path planing problem of UAV as a multi-objective optimization problem, and proposed a new multi-gene structure to describe the path, in which the adaptive adjustment, crossover and mutation strategies were adopted, and the adaptive differential multi-objective optimization algorithm was applied to obtain the optimal solution to avoid obstacles and meet the flight restrictions of UAV. [79] used the minimum spanning tree and adaptive tournament selection to quantify and control the genetic diversity, which solved the problem of premature stagnation. For the obstacle avoidance problem in multi-UAV scenario, [80] used genetic algorithm and evolutionary robot to evolve neural network controller, solving the obstacle avoidance problem of multi-UAV.…”

Learning Robust Policies for Generalized Debris Capture with an Automated Tether-Net System

“…Various ANN fitting (learning) methods have seen demonstrations on robotics and control applications. Popular learning methods include Reinforcement Learning [16], [17], Supervised Learning [18], Imitation Learning [19], Neuroevolution [20], [21], etc., among which the advanced reinforcement learning [22] and neuroevolution [23] methods are directly applicable to launching and wrapping control of tether-net systems. These aforementioned machine learning methods bring capabilities of adapting to system uncertainties, and selecting optimal actions (policies) according to various debris characteristics.…”