Cross-modal and scale-free action representations through enaction

Pitti, Alexandre; Alirezaei, Hassan; Kuniyoshi, Yasuo

doi:10.1016/j.neunet.2009.01.007

Cited by 19 publications

(14 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Here, we categorize them into two groups: nonaction-oriented body representations (body image), and action-oriented ones (body schema). The former body of work employs cross-modal maps that are modified through Hebbian learning applied on individual modalities [96], [134]- [137]. The latter category comprises studies in which the acquired body representations are utilized to coordinate the robot's behavior [138]- [140].…”

Section: Robots As Models Of Biological Body Representationsmentioning

confidence: 99%

Body Schema in Robotics: A Review

Hoffmann

Marques

Arieta

et al. 2010

IEEE Trans. Auton. Mental Dev.

210

101

View full text Add to dashboard Cite

Abstract-How is our body imprinted in our brain? This seemingly simple question is a subject of investigations of diverse disciplines, psychology, and philosophy originally complemented by neurosciences more recently. Despite substantial efforts, the mysteries of body representations are far from uncovered. The most widely used notions-body image and body schema-are still waiting to be clearly defined. The mechanisms that underlie body representations are coresponsible for the admiring capabilities that humans or many mammals can display: combining information from multiple sensory modalities, controlling their complex bodies, adapting to growth, failures, or using tools. These features are also desirable in robots. This paper surveys the body representations in biology from a functional or computational perspective to set ground for a review of the concept of body schema in robotics. First, we examine application-oriented research: how a robot can improve its capabilities by being able to automatically synthesize, extend, or adapt a model of its body. Second, we summarize the research area in which robots are used as tools to verify hypotheses on the mechanisms underlying biological body representations. We identify trends in these research areas and propose future research directions.

show abstract

Section: Robots As Models Of Biological Body Representationsmentioning

confidence: 99%

Body Schema in Robotics: A Review

Hoffmann

Marques

Arieta

et al. 2010

IEEE Trans. Auton. Mental Dev.

210

101

View full text Add to dashboard Cite

show abstract

“…This work pursues several models of the parieto-motor system on which we studied the contributions of motor and spatial development to social cognition [17]. In previous researches, we used spiking neural networks for the detection and learning of synchrony based on the mechanism of spike timing-dependent plasticity [18], either in robotic experiments or in computer simulations [19], [20]. Here, we investigate similar principles using this time the rank-order coding mechanism (ROC) exploited for its rapid spike-based processing [21].…”

Section: Introductionmentioning

confidence: 99%

Gain-field modulation mechanism in multimodal networks for spatial perception

Pitti

Blanchard

Cardinaux

et al. 2012

2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012)

View full text Add to dashboard Cite

Abstract-Seeing is not just done through the eyes, it involves the integration of other modalities such as auditory, proprioceptive and tactile information, to locate objets, persons and also the limbs. We hypothesize that the neural mechanism of gain-field modulation, which is found to process coordinate transform between modalities in the superior colliculus and in the parietal area, plays a key role to build such unified perceptual world. In experiments with a head-neck-eye's robot with a camera and microphones, we study how gain-field modulation in neural networks can serve for transcribing one modality's reference frame into another one (e.g., audio signals into eyes' coordinate). It follows that each modality influences the estimations of the position of a stimulus (multimodal enhancement). This can be used in example for mapping sound signals into retina coordinates for audio-visual speech perception.

show abstract

“…More precisely, the biologically inspired mechanism of spike-timing-dependent plasticity (STDP1920) can serve to detect the contingency at the neural level for categorizing the sensorimotor signals in a situation of interaction with others21. Because STDP permits reinforcement of the synaptic links between synchronous neurons and prediction of long sensorimotor patterns in real time22, the prediction level can serve for contingency detection and self-motion recognition2324.…”

mentioning

confidence: 99%

Spatio-Temporal Tolerance of Visuo-Tactile Illusions in Artificial Skin by Recurrent Neural Network with Spike-Timing-Dependent Plasticity

Pitti¹,

Pugach²,

Gaussier³

et al. 2017

Sci Rep

View full text Add to dashboard Cite

Perceptual illusions across multiple modalities, such as the rubber-hand illusion, show how dynamic the brain is at adapting its body image and at determining what is part of it (the self) and what is not (others). Several research studies showed that redundancy and contingency among sensory signals are essential for perception of the illusion and that a lag of 200–300 ms is the critical limit of the brain to represent one’s own body. In an experimental setup with an artificial skin, we replicate the visuo-tactile illusion within artificial neural networks. Our model is composed of an associative map and a recurrent map of spiking neurons that learn to predict the contingent activity across the visuo-tactile signals. Depending on the temporal delay incidentally added between the visuo-tactile signals or the spatial distance of two distinct stimuli, the two maps detect contingency differently. Spiking neurons organized into complex networks and synchrony detection at different temporal interval can well explain multisensory integration regarding self-body.

show abstract

Cross-modal and scale-free action representations through enaction

Cited by 19 publications

References 41 publications

Body Schema in Robotics: A Review

Body Schema in Robotics: A Review

Gain-field modulation mechanism in multimodal networks for spatial perception

Spatio-Temporal Tolerance of Visuo-Tactile Illusions in Artificial Skin by Recurrent Neural Network with Spike-Timing-Dependent Plasticity

Contact Info

Product

Resources

About