A simple vector-like law for perceptual information combination is also followed by a class of cortical multisensory bimodal neurons

Abstract

Summary An interdisciplinary approach to sensory information combination shows a correspondence between perceptual and neural measures of nonlinear multisensory integration. In psychophysics, sensory information combinations are often characterized by the Minkowski formula, but the neural substrates of many psychophysical multisensory interactions are unknown. We show that audiovisual interactions – for both psychophysical detection threshold data and cortical bimodal neurons – obey similar vector-like Minkowski models, suggesting that cortical bimodal neurons could underlie multisensory perceptual sensitivity. An alternative Bayesian model is not a good predictor of cortical bimodal response. In contrast to cortex, audiovisual data from superior colliculus resembles the ‘City-Block’ combination rule used in perceptual similarity metrics. Previous work found a simple power law amplification rule is followed for perceptual appearance measures and by cortical subthreshold multisensory neurons. The two most studied neural cell classes in cortical multisensory interactions may provide neural substrates for two important perceptual modes: appearance-based and performance-based perception.

Multisensory-inspired modeling and neural correlates for two key binocular interactions

“…As such they are good candidates for being modeled by a weighted average, but which one? A preliminary report from our laboratory found that in each of three tested cases (visual-tactile, audio-tactile and audio-visual), multisensory Between-neurons were well fit by the Schrodinger model and that reliability weighting did not do quite as well 59 . For binocular facilitation we found two candidate neural classes: true binocular neurons and gated binocular neurons.…”

“…Modeling true binocular neurons and comparison to the seemingly analogous case of the bimodal multisensory neurons Just as true binocular neurons can fire to either eye, there are well-studied multisensory 'bimodal' neurons, that fire when either sense (say visual and auditory) are stimulated. These multisensory neurons' spike rates are well fit by the Minkowski model 13 , but are generally less well fit by the power law 28 . This makes sense if neurons are combining responses, however nonlinearly.…”

“…Similar suppressive neurons are found in many multisensory systems (audiovisual, visual-tactile, audio-tactile and visual-infrared) 54,57 , in many species (cats, ferrets, macaques, rattlesnakes) and their perceptual roles also mostly obscure. Recently here has been suggestions that visual-vestibular neurons, many of which are mildly suppressive, might be mimicking a Bayesian weighted average 58 ; and one analysis suggests a particular nonlinear weighted average-invented by www.nature.com/scientificreports/ Erwin Schrödinger to account for binocular appearance-also models mildly suppressive multisensory neurons 59 .…”

“…As MacLeod notes, visual neurons with center/surround receptive fields transmit nonlinearly transformed luminance differences to the brain 7 . This suggests that no mathematical extension of Schrodinger's equation should be necessary to model The special graphing approach, developed for modeling multisensory data 13 plots monocular and binocular neural responses and modeled binocular response as a function of the binocular neural response. Actual binocular neural data plot (by design) on the 45° axis, monocular inputs plot relative to binocular outputs (illustrating relative monocular contributions to outputs), and model deviations from data stand out.…”

“…However, because our results suggest that the 'summation' part of binocular summation might be a misleading misnomer, we will refer -as a theory-neutral term-to 'binocular enhancement' . In vision research 9,10 and in multisensory integration [11][12][13] subadditive combinations of two sensory channels are often studied using the versatile Minkowski equation, In binocular vision it takes the form of Eq. ( 2).…”