Coupling Evolution and Information Theory for Autonomous Robotic Exploration

Abstract

This paper investigates a hybrid two-phase approach toward exploratory behavior in robotics. In a first phase, controllers are evolved to maximize the quantity of information in the sensori-motor datastream generated by the robot. In a second phase, the data acquired by the evolved controllers is used to support an information theory-based controller, selecting the most informative action in each time step. The approach, referred to as EvITE, is shown to outperform both the evolutionary and the information theory-based approaches standalone, in terms of actual exploration of the arena. Further, the EvITE controller features some generality property, being able to efficiently explore other arenas than the one considered during the first evolutionary phase.

Curiosity Search: Producing Generalists by Encouraging Individuals to Continually Explore and Acquire Skills throughout Their Lifetime

“…Another work adopted a similar approach to that proposed in this paper, pressuring agents to express high entropy in the sensorimotor states they visited [ 24 ]. Although that work did not specifically investigate the problem of producing generalists, it did provide a theoretical framework for promoting exploration without any domain knowledge and was later proven to be effective at promoting exploration on a maze navigation task [ 25 ]. It may be worth studying whether their approach also produces generalists or whether Curiosity Search might be made more general by adopting their methodology.…”

Unsupervised identification and recognition of situations for high-dimensional sensori-motor streams

“…The -means algorithm lacks enough sophistication to reliably cluster high dimensional data. Sebag [30] extended this work by collecting the sensori-motor stream of selected controllers. Standard (i.e., not on-line) clustering techniques then discretise the state and action space for subsequent reinforcement learning.…”