Curiosity-driven exploration enhances motor skills of continuous actor-critic learner

Enhancing a Neurocognitive Shared Visuomotor Model for Object Identification, Localization, and Grasping With Learning From Auxiliary Tasks

“…Approaches that rely on deep reinforcement learning [27] require many samples; the resulting training times are often not suitable for physical robot platforms. Therefore, approaches to increase the sample efficiency of (continuous) deep reinforcement learning have focused on sample selection strategies [37] or biologically inspired methods for accelerating the learning process, like curiosity-driven exploration [10], [14] as well as curriculum and incremental learning [7], [36]. A faster approach is to transform the reinforcement learning task into a supervised learning problem by generating fully annotated training samples of visuomotor actions [25].…”

Deep intrinsically motivated continuous actor-critic for efficient robotic visuomotor skill learning

“…In order to allow RL agents to efficiently and meaningfully explore in a sparse-reward world, intrinsically motivated RL methods have been proposed providing the agent a number of intrinsic drives, with artificial curiosity being the most common. Different functions have been defined to design an intrinsic reward for the agent, including Bayesian surprise [16], information gain [17], empowerment [18], prediction error of a learned forward model [6,19,20], predictive learning progress [21,22], and policy value change [23]. In deep RL, Stadie et al and Jaderberg et al also use intrinsic feedback to aid the learning of a target task instead of relying exclusively on sparse extrinsic rewards [6,7].…”

Sensorimotor contingencies as a key drive of development: from babies to robots

“…The work from Oudeyer et al (2005) confirms that a simple robot equipped with what has been called "intelligent adaptive curiosity" can indeed acquire information about its body, at least information of an implicit type --that is, the agent gradually learns to use its body more effectively to explore its environment. Following the idea that understanding one's effects on the environment is crucial for the autonomous development of animals and humans (White 1959;Berlyne, 1960) a variety of work in robotics has focused on the autonomous learning of skills on the basis of the interactions between the body of the artificial agent and the environment, where robots are tested in "simple" reaching or avoidance scenarios (e.g., Santucci et al, 2014a;Hafez et al, 2017;Reinhart, 2017;Hester and Stone, 2017;Tannenberg et al, 2019) or in more complex tasks involving interactions between objects (Da Silva, 2014;Seepanomwan et al, 2017), tool use (Forestier and Oudeyer, 2016) or hierarchical skill learning (Forestier et al, 2017;Colas et al, 2018;Santucci et al, 2019), and even in imitation learning experiments (Duminy et al, 2018). When combined with the use of "goals", intended here as specific states or effects that a system is willing to obtain, curiosity and intrinsic motivation are able to properly guide task selection (Merrick, 2012;Santucci et al, 2016) and reduce the exploration space (Rolf et al, 2010;Baranes and Oudeyer, 2013).…”