Intrinsic motivation and mental replay enable efficient online adaptation in stochastic recurrent networks

Abstract

Autonomous robots need to interact with unknown, unstructured and changing environments, constantly facing novel challenges. Therefore, continuous online adaptation for lifelong-learning and the need of sample-efficient mechanisms to adapt to changes in the environment, the constraints, the tasks, or the robot itself are crucial. In this work, we propose a novel framework for probabilistic online motion planning with online adaptation based on a bio-inspired stochastic recurrent neural network. By using learning signals which mimic the intrinsic motivation signal cognitive dissonance in addition with a mental replay strategy to intensify experiences, the stochastic recurrent network can learn from few physical interactions and adapts to novel environments in seconds. We evaluate our online planning and adaptation framework on an anthropomorphic KUKA LWR arm. The rapid online adaptation is shown by learning unknown workspace constraints sample-efficiently from few physical interactions while following given way points.

Impact of transformational leadership on work performance, burnout and social loafing: a mediation model

“…Intrinsic motivation may be defined as "the doing of an activity for its inherent satisfactions rather than for some separable consequence but it is rare for employees to experience intrinsic motivation in all of their tasks" [76]. Intrinsic motivation is generated for self-developing attributes that refers to make an individual ready to be the part of learning procedure without having interests of extrinsic rewards [83]. Intrinsic motivation is basically the degree of an individual's interest in a task completion and how he engages himself in work [3].…”

Sensorimotor Self-organization via Circular-Reactions

“…The neural networks were trained via reinforcement learning where both the visual inputs and the motor system were represented by vectors. Tanneberg et al ( 2019 ) implemented a stochastic recurrent network to refine the end-effector's motion trajectory to avoid the obstacles on the way during reaching. In their study, visual inputs were not used.…”

Sensorimotor Contingencies as a Key Drive of Development: From Babies to Robots

“…Work on open-ended learning (Baldassarre and Mirolli, 2013) confirms that a simple agent equipped with what has been called “intelligent adaptive curiosity” can indeed acquire information about the effects that it can generate in the environment and leverage these sensorimotor contingencies to learn or refine its skills. Following the idea that understanding one’s effects on the environment is crucial for the autonomous development of animals and humans (White, 1959; Berlyne, 1960) different work in robotics has focused on the autonomous learning of skills on the basis of the interactions between the body of the artificial agent and the environment, where robots are tested in “simple” reaching or avoidance scenarios (e.g., Santucci et al, 2014; Hafez et al, 2017; Hester and Stone, 2017; Reinhart, 2017; Tanneberg et al, 2019) or in more complex tasks involving interactions between objects (da Silva et al, 2014; Seepanomwan et al, 2017), tool use or hierarchical skill learning (Forestier et al, 2017; Colas et al, 2018; Santucci et al, 2019), and even in imitation learning experiments (Duminy et al, 2018). When combined with the use of “goals,” intended here as specific states or effects that a system is trying to attain, curiosity and intrinsic motivation are able to properly guide task selection (Merrick, 2012; Santucci et al, 2016) and reduce the exploration space (Rolf et al, 2010; Baranes and Oudeyer, 2013).…”