Hyper-Learning Algorithms for Online Evolution of Robot Controllers

Abstract

A long-standing goal in artificial intelligence and robotics is synthesising agents that can effectively learn and adapt throughout their lifetime. One open-ended approach to behaviour learning in autonomous robots is online evolution , which is part of the evolutionary robotics field of research. In online evolution approaches, an evolutionary algorithm is executed on the robots during task execution, which enables continuous optimisation and adaptation of behaviour. Despite the potential for automatic behaviour learning, online evolution has not been widely adopted because it often requires several hours or days to synthesise solutions to a given task. In this respect, research in the field has failed to develop a prevalent algorithm able to effectively synthesise solutions to a large number of different tasks in a timely manner. Rather than focusing on a single algorithm, we argue for more general mechanisms that can combine the benefits of different algorithms to increase the performance of online evolution of robot controllers. We conduct a comprehensive assessment of a novel approach called online hyper-evolution (OHE). Robots executing OHE use the different sources of feedback information traditionally associated with controller evaluation to find effective evolutionary algorithms during task execution. First, we study two approaches: OHE-fitness, which uses the fitness score of controllers as the criterion to select promising algorithms over time, and OHE-diversity, which relies on the behavioural diversity of controllers for algorithm selection. We then propose a novel class of techniques called OHE-hybrid, which combine diversity and fitness to search for suitable algorithms. In addition to their effectiveness at selecting suitable algorithms, the different OHE approaches are evaluated for their ability to construct algorithms by controlling which algorithmic components should be employed for controller generation (e.g., mutation, crossover, among others), an unprecedented approach in evolutionary robotics. Results show that OHE (i) facilitates the evolution of controllers with high performance, (ii) can increase effectiveness at different stages of evolution by combining the benefits of multiple algorithms over time, and (iii) can be effectively applied to construct new algorithms during task execution. Overall, our study shows that OHE is a powerful new paradigm that allows robots to improve their learning process as they operate in the task environment.

Evolutionary online behaviour learning and adaptation in real robots

“…While the online evolutionary process is able to consistently synthesize a set of solutions to the homing task, there are different options to potentially increase the performance of evolution in terms of the percentage of successful controllers. In recent simulation-based contributions, we have developed two complementary approaches, namely: (i) a racing technique to cut short the evaluation of poor controllers based on the task performance of past controllers [ 29 ] and (ii) a novel paradigm called online hyper-evolution [ 30 , 31 ], which can both combine the benefits of different algorithms for controller generation over time, and construct algorithms by selecting which algorithmic components should be employed for controller generation (e.g. mutation, crossover, among others).…”

Evolutionary Synthesis of Sensing Controllers for Voxel-based Soft Robots

“…In an attempt of quantifying the result of this qualitative analysis, we devised a way of systematically capturing and describing the behaviors of the VSR-similar procedures have been already used for analyzing the behavior of robots with evolved controllers, e.g., in Silva et al (2017). We proceeded as follows.…”