Incremental Embodied Chaotic Exploration of Self-Organized Motor Behaviors with Proprioceptor Adaptation

Abstract

This paper presents a general and fully dynamic embodied artificial neural system, which incrementally explores and learns motor behaviors through an integrated combination of chaotic search and reflex learning. The former uses adaptive bifurcation to exploit the intrinsic chaotic dynamics arising from neuro-body-environment interactions, while the latter is based around proprioceptor adaptation. The overall iterative search process formed from this combination is shown to have a close relationship to evolutionary methods. The architecture developed here allows realtime goal-directed exploration and learning of the possible motor patterns (e.g., for locomotion) of embodied systems of arbitrary morphology. Examples of its successful application to a simple biomechanical model, a simulated swimming robot, and a simulated quadruped robot are given. The tractability of the biomechanical systems allows detailed analysis of the overall dynamics of the search process. This analysis sheds light on the strong parallels with evolutionary search.

The chaotic dynamics and multistability of two coupled Fitzhugh–Nagumo model neurons

“…In biological terms, in this state an individual FHN is at the border of two important types of oscillatory (CPG) behaviour: half-centre (requiring a reciprocally linked partner FHN) and pacemaker (having intrinsic oscillatory dynamics of its own). Dynamics poised on this border can be exploited in a powerful way in the development, learning and generation of motor behaviours Kuniyoshi et al (2007); Shim and Husbands (2015) and the analysis in this paper will help to refine such research. For instance, it was demonstrated that chaotic dynamics emerging spontaneously from interactions between neural circuitry, bodies, and environments can be used to power a kind of search process (chaotic search) enabling an embodied system to explore its own possible motor behaviours (Kuniyoshi and Suzuki 2004).…”

Fusing autonomy and sociality via embodied emergence and development of behaviour and cognition from fetal period

“…Recently, Shim & Husbands [55,56] greatly enhanced the generality and effectiveness of the framework by introducing sensory adaptation and dynamic order parameters. Der & Martius [57] proposed an alternative framework using a novel synaptic learning mechanism with very similar aims.…”

Darwinian Dynamics of Embodied Chaotic Exploration

“…The whole process can be interpreted as an iterative, continuous, and deterministic version of stochastic trial-and-error search (for fitter attractors within a given state space), which exploits the intrinsic chaotic behavior of the system. A visualization of the attractor landscape deformation of a simple mass-spring system (top) and the exploration behavior of a three-armed finned swimmer (bottom) [17]. Each spring-damper complex represents the simplified Hill type muscle, and the torsional version of the same muscle model was used for the swimming robot.…”

“…This was further extended to explore beyond the fixed set of locomotor behaviors that is initially given by the physical embodiment by allowing the iterative use of embodied chaotic exploration (ECE) through the addition of proprioceptor adaptation [17]. These studies raise an interesting idea about the relationship between fitness-directed ECE and Darwinian neural dynamics [3] and we suggest there is a strong direct analogy between ECE and estimation of distribution algorithms (EDAs) [11,9], a well-established class of evolutionary algorithms, and that ECE can be viewed as a specialized form of Darwinian neural dynamics.…”