Predictive influences of auditory information on resolution of visual competition were investigated using music, whose visual symbolic notation is familiar only to those with musical training. Results from two experiments using different experimental paradigms revealed that melodic congruence between what is seen and what is heard impacts perceptual dynamics during binocular rivalry. This bisensory interaction was observed only when the musical score was perceptually dominant, not when it was suppressed from awareness, and it was observed only in people who could read music. Results from two ancillary experiments showed that this effect of congruence cannot be explained by differential patterns of eye movements or by differential response sluggishness associated with congruent score/melody combinations. Taken together, these results demonstrate robust audiovisual interaction based on high-level, symbolic representations and its predictive influence on perceptual dynamics during binocular rivalry.music | binocular rivalry | multisensory interaction | visual awareness | audio-visual congruence V ision can be confusing, particularly when the retinal images inaugurating vision are ambiguous or underspecified with respect to their origins (1, 2). In some situations, confusion can be resolved by reliance on contextual information (3), prior experience (4), expectations (5), and/or implicit knowledge about structural regularities in our visual environment (6). In other situations, however, confusion resists resolution, in which case one can directly experience the dynamical nature of competition as conflicting perceptual interpretations vie for dominance (7). Among the visual phenomena exhibiting this behavior, none is more compelling than binocular rivalry (8).When the two eyes view dissimilar monocular stimuli, stable binocular single vision gives way to interocular competition, characterized by unpredictable fluctuations in visual awareness of the rival stimuli (9, 10). Rivalry offers a way to study the inferential nature of perception when the brain is confronted with conflicting sensory evidence (11,12). Within this context, recent studies imply that rivalry dynamics are governed by influences related to the likelihood of competing perceptual interpretations, such as motor actions (13), affective connotation (14-16), familiarity (17), and concomitant sensory input from other modalities (18-20). Here we examined the limits of susceptibility of rivalry to predictive influences by asking whether binocular rivalry dynamics depend on multisensory congruence between abstract representations familiar to individuals with expertise in that domain of abstraction. The domain we selected was music, for three reasons: (i) musical notation is symbolic; (ii) only people versed in music can decipher that notation; and (iii) melodic structure can be experienced via sight or sound.
ResultsExperiment 1: Extended Tracking of Binocular Rivalry. Experiment 1 was implemented as a conventional binocular rivalry task in which participants...