Coupling BCM and Neural Fields for the Emergence of Self-organization Consensus

Abstract

Human beings interact with the environment through different modalities, i.e. perceptions and actions, processed in the cortex by dedicated brain areas. These areas are self-organized, so that spatially close neurons are sensitive to close stimuli, providing generalization from previously learned examples. Although perceptive flows are picked up by different spatially separated sensors, their processings are not isolated. On the contrary, they are constantly interacting, as illustrated by the McGurk effect. When the auditory stimulus /ba/ and the /ga/ lip movement are presented simultaneously, people perceive a /da/, which does not correspond to any of the stimuli. Merging several stimuli into one multimodal perception reduces ambiguities and noises and is essential to interact with the environment. This article proposes a model for modality association, inspired by the biological properties of the cortex. The model consists of modality maps interacting through an associative map to raise a consistent multimodal perception of the environment. We propose the coupling of BCM learning rule and neural maps to obtain the decentralized and unsupervised self-organization of each modal map influenced by the multisensory context. We obtain local self-organization of modal maps with various inputs and discretizations.

SOMMA: Cortically inspired paradigms for multimodal processing

“…In previous articles [25,26], we proposed cortical maps for monomodal processing using the coupling of BCMu learning rule with neural fields. In this article, we introduce multimodal processing which is based on the addition of a cortical layer that receives topographic connections between maps to modulate computation in each map.…”

“…Sensory and perceptive layers as well as their coupling were already presented in [25,26]. In this article, we introduce the cortical layer and its incoming multimodal topographic connections.…”

“…By the way, this autonomous emergence improves the coupling with the perceptive layer using CNFT (please refer to [25] for technical arguments). • Because of the bump shape of perceptive activities, close columns receive close modulation signals.…”

“…As a result, sensory layer of close columns will have close discriminations. Thus, perceptive activities act as a decentralized neighborhood function, ensuring map selforganization [25]. • Because of the resonance mechanism, perceptive activity bumps are forced to be located at the same position in all maps.…”

“…Each column learns a discrimination to a stimulus of the input flow with the BCMu learning rule [26]. These discriminations are self-organized in each map thanks to the coupling with neural fields used as a neighborhood function [25]. Learning and computation in each map is influenced by other modalities thanks to bidirectional topographic connections between all maps.…”