An Enactive approach to autonomous agent and robot learning

Abstract

A novel way to model an agent interacting with an environment is introduced, called an Enactive Markov Decision Process (EMDP). An EMDP keeps perception and action embedded within sensorimotor schemes rather than dissociated. Instead of seeking a goal associated with a reward, as in reinforcement learning, an EMDP agent is driven by two forms of self-motivation: successfully enacting sequences of interactions (autotelic motivation), and preferably enacting interactions that have predefined positive values (interactional motivation). An EMDP learning algorithm is presented. Results show that the agent develops a rudimentary form of self-programming, along with active perception as it learns to master the sensorimotor contingencies afforded by its coupling with the environment.

Analyzing Enactive AI implementations on Enactive AI terms

“…Olivier Georgeon has several works on RL and the Enactive approach, [19,20] to cite few, but only in [14] he addresses the enactive paradigm. In [14] Georgeon, Wolf and Gay successfully apprehend the Enactive concept. The work does a good contribution to the Enactive field in putting the agent in a more active role in the sensorimotor loop.…”

“…We chose a representative one from the group of related works and will present next. There is one work by Rafik Hadfi [13] we found that talks about enactivism and uses the work of Olivier Georgeon et al [14] (discussed next) to run an experiment, but Hadfi is much more interested in comparing Enactive Learning with Reinforcement Learning than making any contribution to advance enactivism, and his work fails to address fundamental points of enactivism. Before discussing the works of implementation we talk about another work that is similar to ours and we state the differences.…”

“…An Enactive Approach to Autonomous Agent and Robot Learning Georgeon, Wolf and Gay [14] propose a modification in Markovian Decision…”

“…In a formal way, EMDP is a "tuple (S, I, q, v) in which S is the set of environment states; I is the set of primitive interactions offered by the coupling between the agent and the environment; q is a probability distribution such that q(s t+1 ∥s t , i t ) gives the probability that the environment transitions to state s t+1 ∈ S when the agent chooses interaction i t ∈ I in state s t ; and v is a probability distribution such that v(e t ∥s t , i t ) gives the probability that the agent receives the input e t ∈ I after choosing it in state s t " [14]. The EMDP cycle consists of choosing an intended interaction, transiting to a new state, generating the enacted interaction and repeat.…”

CASH only: Constitutive autonomy through motorsensory self-programming

“… Video 2 (https://youtu.be/t1RO5S4mBEY) demonstrates the same algorithm implemented in an e-puck robot (Georgeon, Wolf & Gay 2013).  Video 3 (https://youtu.be/91kKzybt8XY) demonstrates the same algorithm used to control distal motorsensory schemes (Georgeon, Cohen & Cordier 2011).…”