Models and processes of multisensory cue combination

Seilheimer, Robert L; Rosenberg, Ari; Angelaki, Dora E.

doi:10.1016/j.conb.2013.11.008

Cited by 97 publications

(91 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such illusions are perceived by several primate species as well as pigeons (29)(30)(31), suggesting that perspective-sensitive cells that disregard disparity may be widely found in visual animals. These cells may also be important for estimating the 3D orientation of distal objects (where disparity cues are less reliable) (6), as well as calibrating orientation estimates based on texture and disparity cues (32)(33)(34). Interestingly, some cells that were sensitive to both texture and disparity cues only signaled the texturedefined slant (e.g., Fig.…”

Section: Discussionmentioning

confidence: 99%

“…Another open question is how neurons are able to combine sensory signals according to their reliability. Recent work suggests this ability requires a computation called divisive normalization in which the response of each neuron is normalized by a measure of the population activity (26,34,38,39). Future experiments can be designed to test whether divisive normalization accounts for the integration of texture and disparity cues in CIP, and if individual neurons can reweight these cues to account for changes in viewing geometry (6,25,26).…”

Section: Discussionmentioning

confidence: 99%

“…Together, these results suggest the existence of two mechanisms contributing to the visual encoding of 3D orientation: reliability-based cue weighting and calibration of different visual signals. In future studies, it will be important to determine how cue weighting and cue calibration (33,34) together ensure that 3D representations of the environment are both reliable and accurate.…”

Section: Discussionmentioning

confidence: 99%

See 2 more Smart Citations

Reliability-dependent contributions of visual orientation cues in parietal cortex

Rosenberg

Angelaki

2014

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

View full text Add to dashboard Cite

Creating accurate 3D representations of the world from 2D retinal images is a fundamental task for the visual system. However, the reliability of different 3D visual signals depends inherently on viewing geometry, such as how much an object is slanted in depth. Human perceptual studies have correspondingly shown that texture and binocular disparity cues for object orientation are combined according to their slant-dependent reliabilities. Where and how this cue combination occurs in the brain is currently unknown. Here, we search for neural correlates of this property in the macaque caudal intraparietal area (CIP) by measuring slant tuning curves using mixed-cue (texture + disparity) and cue-isolated (texture or disparity) planar stimuli. We find that texture cues contribute more to the mixed-cue responses of CIP neurons that prefer larger slants, consistent with theoretical and psychophysical results showing that the reliability of texture relative to disparity cues increases with slant angle. By analyzing responses to binocularly viewed texture stimuli with conflicting texture and disparity information, some cells that are sensitive to both cues when presented in isolation are found to disregard one of the cues during cue conflict. Additionally, the similarity between texture and mixed-cue responses is found to be greater when this cue conflict is eliminated by presenting the texture stimuli monocularly. The present findings demonstrate reliability-dependent contributions of visual orientation cues at the level of the CIP, thus revealing a neural correlate of this property of human visual perception.vision | 3D orientation | perspective | reliability | cue combination T ransforming 2D retinal images into accurate 3D representations of the world is a fundamental, albeit complex, problem the brain must solve. The computation of 3D object orientation is essential to this process and necessary for a wide range of behaviors, including object recognition (1), reaching (2), and grasping (3, 4). Multiple signals, including texture (available monocularly) and binocular disparity, are used to determine 3D orientation (5, 6). Single-unit (7-10) and functional MRI (fMRI) (11-13) studies indicate that different orientation cues are combined in high-level cortical areas. Object orientation is often described using angular variables called slant (rotation about an axis perpendicular to the line of sight) and tilt (rotation about an axis parallel to the line of sight) (14, 15) (Fig. S1). As a consequence of perspective geometry, which determines how a scene projects onto each retina (16), the reliability of texture cues for 3D orientation increases with slant (i.e., as depth variation increases) (17) (Fig. 1A). In contrast, the reliability of disparity cues is largely independent of slant (18). Thus, if robust orientation estimates are created by combining texture and disparity cues according to their reliabilities, the relative contributions of the cues will depend on the object's slant. Human perceptual studies corresponding...

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Reliability-dependent contributions of visual orientation cues in parietal cortex

Rosenberg

Angelaki

2014

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Divisive normalization inherently reflects the E/I balance, and is implicated in a wide range of processes ranging from sensory encoding to decision making (15)(16)(17). Using neural network simulations, we show that a reduction in the amount of inhibition that occurs through divisive normalization can account for perceptual consequences reported in the disorder, providing a bridge between an E/I imbalance and the behavioral characteristics of autism.…”

Section: Bayesian Inferencementioning

confidence: 91%

“…One possibility is that population activity is decoded to determine the most probable stimulus, and the appropriate behavioral response is then selected accordingly (17,38,42). This strategy captures the intuition that greater activity in a subpopulation of neurons with similar response properties leads to greater certainty about the stimulus (SI Appendix).…”

Section: Simulation 1: Visual Spatial Suppressionmentioning

confidence: 99%

A computational perspective on autism

Rosenberg

Patterson

Angelaki

2015

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

149

151

View full text Add to dashboard Cite

Autism is a neurodevelopmental disorder that manifests as a heterogeneous set of social, cognitive, motor, and perceptual symptoms. This system-wide pervasiveness suggests that, rather than narrowly impacting individual systems such as affection or vision, autism may broadly alter neural computation. Here, we propose that alterations in nonlinear, canonical computations occurring throughout the brain may underlie the behavioral characteristics of autism. One such computation, called divisive normalization, balances a neuron's net excitation with inhibition reflecting the overall activity of the neuronal population. Through neural network simulations, we investigate how alterations in divisive normalization may give rise to autism symptomatology. Our findings show that a reduction in the amount of inhibition that occurs through divisive normalization can account for perceptual consequences of autism, consistent with the hypothesis of an increased ratio of neural excitation to inhibition (E/I) in the disorder. These results thus establish a bridge between an E/I imbalance and behavioral data on autism that is currently absent. Interestingly, our findings implicate the context-dependent, neuronal milieu as a key factor in autism symptomatology, with autism reflecting a less "social" neuronal population. Through a broader discussion of perceptual data, we further examine how altered divisive normalization may contribute to a wide array of the disorder's behavioral consequences. These analyses show how a computational framework can provide insights into the neural basis of autism and facilitate the generation of falsifiable hypotheses. A computational perspective on autism may help resolve debates within the field and aid in identifying physiological pathways to target in the treatment of the disorder.A utism is a neurodevelopmental disorder that is dramatically increasing in prevalence (Fig. 1). Recent data place the number of children being diagnosed with autism in the United States at 1 in 68, more than doubling in the last decade (1-4). The disorder is highly pervasive, affecting individuals at cognitive, motor, and perceptual levels. It is furthermore a "spectrum disorder," with symptoms that manifest in varying degrees across individuals. This heterogeneity presents significant challenges to establishing a comprehensive characterization of the disorder.Research investigating the genetic and molecular basis of autism implicates over 100 genes (5), many of which are involved in synaptic development and function (6-8). As such, one prominent hypothesis is that autism arises from a neurophysiological excitation-to-inhibition (E/I) imbalance (9, 10). However, the connection between an E/I imbalance and the behavioral characteristics of the disorder remains unclear. Considering the pervasive nature of autism, and the covariance of loosely related symptoms (11-14), one possibility is that an E/I imbalance widely affects neural computation, in turn giving rise to the broad behavioral symptoms recognized as autism.Here,...

show abstract

The intraparietal sulcus governs multisensory integration of audiovisual information based on task difficulty

Regenbogen

Seubert

Johansson

et al. 2017

Human Brain Mapping

View full text Add to dashboard Cite

Object recognition benefits maximally from multimodal sensory input when stimulus presentation is noisy, or degraded. Whether this advantage can be attributed specifically to the extent of overlap in object-related information, or rather, to object-unspecific enhancement due to the mere presence of additional sensory stimulation, remains unclear. Further, the cortical processing differences driving increased multisensory integration (MSI) for degraded compared with clear information remain poorly understood. Here, two consecutive studies first compared behavioral benefits of audio-visual overlap of object-related information, relative to conditions where one channel carried information and the other carried noise. A hierarchical drift diffusion model indicated performance enhancement when auditory and visual object-related information was simultaneously present for degraded stimuli. A subsequent fMRI study revealed visual dominance on a behavioral and neural level for clear stimuli, while degraded stimulus processing was mainly characterized by activation of a frontoparietal multisensory network, including IPS. Connectivity analyses indicated that integration of degraded object-related information relied on IPS input, whereas clear stimuli were integrated through direct information exchange between visual and auditory sensory cortices. These results indicate that the inverse effectiveness observed for identification of degraded relative to clear objects in behavior and brain activation might be facilitated by selective recruitment of an executive cortical network which uses IPS as a relay mediating crossmodal sensory information exchange.

show abstract

Models and processes of multisensory cue combination

Cited by 97 publications

References 62 publications

Reliability-dependent contributions of visual orientation cues in parietal cortex

Reliability-dependent contributions of visual orientation cues in parietal cortex

A computational perspective on autism

The intraparietal sulcus governs multisensory integration of audiovisual information based on task difficulty

Contact Info

Product

Resources

About