Development of a Bayesian Estimator for Audio-Visual Integration: A Neurocomputational Study

Abstract

The brain integrates information from different sensory modalities to generate a coherent and accurate percept of external events. Several experimental studies suggest that this integration follows the principle of Bayesian estimate. However, the neural mechanisms responsible for this behavior, and its development in a multisensory environment, are still insufficiently understood. We recently presented a neural network model of audio-visual integration (Neural Computation, 2017) to investigate how a Bayesian estimator can spontaneously develop from the statistics of external stimuli. Model assumes the presence of two unimodal areas (auditory and visual) topologically organized. Neurons in each area receive an input from the external environment, computed as the inner product of the sensory-specific stimulus and the receptive field synapses, and a cross-modal input from neurons of the other modality. Based on sensory experience, synapses were trained via Hebbian potentiation and a decay term. Aim of this work is to improve the previous model, including a more realistic distribution of visual stimuli: visual stimuli have a higher spatial accuracy at the central azimuthal coordinate and a lower accuracy at the periphery. Moreover, their prior probability is higher at the center, and decreases toward the periphery. Simulations show that, after training, the receptive fields of visual and auditory neurons shrink to reproduce the accuracy of the input (both at the center and at the periphery in the visual case), thus realizing the likelihood estimate of unimodal spatial position. Moreover, the preferred positions of visual neurons contract toward the center, thus encoding the prior probability of the visual input. Finally, a prior probability of the co-occurrence of audio-visual stimuli is encoded in the cross-modal synapses. The model is able to simulate the main properties of a Bayesian estimator and to reproduce behavioral data in all conditions examined. In particular, in unisensory conditions the visual estimates exhibit a bias toward the fovea, which increases with the level of noise. In cross modal conditions, the SD of the estimates decreases when using congruent audio-visual stimuli, and a ventriloquism effect becomes evident in case of spatially disparate stimuli. Moreover, the ventriloquism decreases with the eccentricity.

From multisensory integration in peripersonal space to bodily self‐consciousness: from statistical regularities to statistical inference

“…A current limitation of Bayesian models, however, is a profound lack in understanding their neural instantiation, in particular from a physiological perspective or even a biological plausibility standpoint, (although novel neural network modeling is rapidly narrowing this gap in knowledge). Importantly, the fact that Bayesian models may account for findings including and surpassing those detailed by the MLE, yet this latter approach and not the former is firmly grounded in the brain, should not be a deterrent, but an incentive for future research.…”

Emergence of opposite neurons in a decentralized firing-rate model of multisensory integration

“…The assumption of Gaussian or von Mises shaped feedforward connections is 161 usually assumed in multisensory integration models, and we show that the same shape 162 can naturally come out in our learning model [9,24,25,29,30]. Since the model is 4D-F.…”

Emergence of opposite neurons in a firing-rate model of multisensory integration

“…The assumption of Gaussian or von Mises shaped feedforward weights is usually assumed in multisensory integration models, and we show that our model can learn feedforward weights that have the same shape [5,6,23,24,22].…”

“…Some studies of ventriloquism have explored the learning of the equivalent of congruent neurons in a similar decentralized model [23,24,22], but they assume a priori topographic organization of the multisensory neurons before learning. Here, no such assumption is made, and topographic organization naturally emerges via a Kohonen map-like mechanism [17].…”

“…In fact, if such a delay is removed, opposite neurons cannot be learned well. We also tried using the anti-Hebbian learning rule introduced by Földiák, which takes the form ∆w ij = −α(r i r j − p 2 ), for some fixed constant p. While opposite neurons can still be learned, the shape of the receptive fields can no longer be well-approximated by Gaussian or von-Mises distributions, an assumption in some decentralized models of multisensory integration [5,6,23,24,22].…”