Intrinsic Motivation Systems for Autonomous Mental Development

Abstract

Abstract-Exploratory activities seem to be intrinsically rewarding for children and crucial for their cognitive development. Can a machine be endowed with such an intrinsic motivation system? This is the question we study in this paper, presenting a number of computational systems that try to capture this drive towards novel or curious situations. After discussing related research coming from developmental psychology, neuroscience, developmental robotics, and active learning, this paper presents the mechanism of Intelligent Adaptive Curiosity, an intrinsic motivation system which pushes a robot towards situations in which it maximizes its learning progress. This drive makes the robot focus on situations which are neither too predictable nor too unpredictable, thus permitting autonomous mental development. The complexity of the robot's activities autonomously increases and complex developmental sequences self-organize without being constructed in a supervised manner. Two experiments are presented illustrating the stage-like organization emerging with this mechanism. In one of them, a physical robot is placed on a baby play mat with objects that it can learn to manipulate. Experimental results show that the robot first spends time in situations which are easy to learn, then shifts its attention progressively to situations of increasing difficulty, avoiding situations in which nothing can be learned. Finally, these various results are discussed in relation to more complex forms of behavioral organization and data coming from developmental psychology.

Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents

“…Maybe agents should not be exploring in terms of joystick movements, but in terms of object configurations and game levels. Finally, for intrinsically difficult games, agents may need some form of intrinsic motivation (Barto, 2013;Oudeyer, Kaplan, & Hafner, 2007) to keep playing despite the apparent impossibility of scoring in the game.…”

TeXDYNA: Hierarchical Reinforcement Learning in Factored MDPs

“…The main issue for future work consists in replacing the basic -greedy exploration strategy by a more sophisticated policy, either along the "optimism in the face of uncertainty" line [14] or based on adding internal motivations such as an artificial curiosity process [15,13] into the framework. Then the application of TeXDYNA to a robotics problem will be the matter of a more experimental work.…”

Beyond black-box optimization: a review of selective pressures for evolutionary robotics

“…Novelty search [106] is a possible starting point, but it raises a new question: in what space do we measure novelty? Another approach is to define and formalize interestingness and define a curiosity-driven search process [160,149] or to rely on more indirect selective pressures as in co-evolution [172,64,65] or environment-driven pressures [27]. Considering the search process as a creative process also raises another important question: on what scientific method do we rely?…”