CHIME: An Adaptive Hierarchical Representation for Continuous Intrinsically Motivated Exploration

Abstract

For a lifelong learning, robots need to learn and adapt to the environment to complete multiple tasks ranging from low-level to high-level tasks, using simple actions or complex ones. Current intrinsically motivated solutions often rely on fixed representations of this environment to define possible tasks, limiting the possibility to adapt to new or changing ones. We propose an algorithm that is able to autonomously 1) self-discover tasks to learn in its environment 2) discover the relationship between tasks to leverage its acquired knowledge on low-level tasks to solve high-level tasks 3) devise a sequence of policies of unbounded length to complete the tasks. Our algorithm, named Continual Hierarchical Intrinsically Motivated Exploration (CHIME), uses planning to build chains of actions, the learning of a hierarchical representation of tasks to reuse low-level skills for high-level tasks and intrinsically-motivated goal babbling to discover new subtasks and orient its learning in its high-dimensional continuous environment. To highlight the features of CHIME, we implement it in a simulated mobile robot where it can move and place objects.

Hierarchical Affordance Discovery using Intrinsic Motivation

“…We decide to base our proposition on the CHIME algorithm and adapt their learning algorithm to the affordance learning problem by using sensorimotor features. Whereas in [13], CHIME could not generalise its skills to new objects, our algorithm is capable to generalise to new objects, as its learning is based on sensory features.…”

“…Our algorithm is based on the CHIME algorithm [13]. Both are iterative and active learning algorithm that learn by episodes, but unlike CHIME our algorithm is designed to consider visual properties during its learning process.…”

“…This algorithm uses intrinsic motivation to guide its exploration. It is based on the CHIME algorithm [13], itself inspired by the SAGG-RIAC algorithm [1].…”

“…More recently, methods using intrinsic motivation to build a hierarchy of interrelated skills has been proposed. Firstly by using a pre-programmed and static hierarchy [7], then by learning it through exploration for robotic arms [5] or mobile robots [13]. The latter provides an algorithm, CHIME, that uses intrinsic motivation to guide its exploration.…”