The perception of self-motion direction, or heading, relies on integration of multiple sensory cues, especially from the visual and vestibular systems. However, the reliability of sensory information can vary rapidly and unpredictably, and it remains unclear how the brain integrates multiple sensory signals given this dynamic uncertainty. Human psychophysical studies have shown that observers combine cues by weighting them in proportion to their reliability, consistent with statistically optimal integration schemes derived from Bayesian probability theory. Remarkably, because cue reliability is varied randomly across trials, the perceptual weight assigned to each cue must change from trial to trial. Dynamic cue reweighting has not been examined for combinations of visual and vestibular cues, nor has the Bayesian cue integration approach been applied to laboratory animals, an important step toward understanding the neural basis of cue integration. To address these issues, we tested human and monkey subjects in a heading discrimination task involving visual (optic flow) and vestibular (translational motion) cues. The cues were placed in conflict on a subset of trials, and their relative reliability was varied to assess the weights that subjects gave to each cue in their heading judgments. We found that monkeys can rapidly reweight visual and vestibular cues according to their reliability, the first such demonstration in a nonhuman species. However, some monkeys and humans tended to over-weight vestibular cues, inconsistent with simple predictions of a Bayesian model. Nonetheless, our findings establish a robust model system for studying the neural mechanisms of dynamic cue reweighting in multisensory perception.

Integration of multiple sensory cues is essential for precise and accurate perception and behavioral performance, yet the reliability of sensory signals can vary across modalities and viewing conditions. Human observers typically employ the optimal strategy of weighting each cue in proportion to its reliability, but the neural basis of this computation remains poorly understood. We trained monkeys to perform a heading discrimination task from visual and vestibular cues, varying cue reliability at random. Monkeys appropriately placed greater weight on the more reliable cue, and population decoding of neural responses in area MSTd nicely predicted behavioral cue weighting, including modest deviations from optimality. We further show that the mathematical combination of visual and vestibular inputs by single neurons is generally consistent with recent theories of optimal probabilistic computation in neural circuits. These results provide direct evidence for a neural mechanism mediating a simple and widespread form of statistical inference.

SUMMARY Responses of neurons in early visual cortex change little with training, and appear insufficient to account for perceptual learning. Behavioral performance, however, relies on population activity, and the accuracy of a population code is constrained by correlated noise among neurons. We tested whether training changes interneuronal correlations in the dorsal medial superior temporal area, which is involved in multisensory heading perception. Pairs of single units were recorded simultaneously in two groups of subjects: animals trained extensively in a heading discrimination task, and “naïve” animals that performed a passive fixation task. Correlated noise was significantly weaker in trained versus naïve animals, which might be expected to improve coding efficiency. However, we show that the observed uniform reduction in noise correlations leads to little change in population coding efficiency when all neurons are decoded. Thus, global changes in correlated noise among sensory neurons may be insufficient to account for perceptual learning.

Humans and monkeys use both vestibular and visual motion (optic flow) cues to discriminate their direction of self-motion during navigation. A striking property of heading perception from optic flow is that discrimination is most precise when subjects judge small variations in heading around straight ahead, whereas thresholds rise precipitously when subjects judge heading around an eccentric reference. We show that vestibular heading discrimination thresholds in both humans and macaques also show a consistent, but modest, dependence on reference direction. We used computational methods (Fisher information, maximum likelihood estimation, and population vector decoding) to show that population activity in area MSTd predicts the dependence of heading thresholds on reference eccentricity. This dependence arises because the tuning functions for most neurons have a steep slope for directions near straight forward. Our findings support the notion that population activity in extrastriate cortex limits the precision of both visual and vestibular heading perception.

The richness of perceptual experience, as well as its usefulness for guiding behavior, depends upon the synthesis of information across multiple senses. Recent decades have witnessed a surge in our understanding of how the brain combines sensory signals, or cues. Much of this research has been guided by one of two distinct approaches, one driven primarily by neurophysiological observations, the other guided by principles of mathematical psychology and psychophysics. Conflicting results and interpretations have contributed to a conceptual gap between psychophysical and physiological accounts of cue integration, but recent studies of visual-vestibular cue integration have narrowed this gap considerably.