Lip Reading Without Awareness

Plass, John; Guzman-Martinez, Emmanuel; Ortega, Laura; Grabowecky, Marcia; Suzuki, Satoru

doi:10.1177/0956797614542132

Cited by 25 publications

(32 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our results add to the body of evidence showing that visual awareness during interocular competition is influenced by nonvisual information conveyed by other sensory modalities (35)(36)(37)(38)(39)(40)(41). Our results go further, however, by demonstrating robust AV interaction based on high-level, symbolic representations, i.e., musical notation and melodic structure.…”

Section: Discussionsupporting

confidence: 76%

“…Our results go further, however, by demonstrating robust AV interaction based on high-level, symbolic representations, i.e., musical notation and melodic structure. It is not surprising that this robust interaction is enjoyed only by individuals with expertise acquired through musical training, in contrast, for example, to the performance enhancement exhibited by virtually all people when listening to speech sounds while looking at lip movements (35,39).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Melodic sound enhances visual awareness of congruent musical notes, but only if you can read music

Lee

Blake

Kim

et al. 2015

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Predictive influences of auditory information on resolution of visual competition were investigated using music, whose visual symbolic notation is familiar only to those with musical training. Results from two experiments using different experimental paradigms revealed that melodic congruence between what is seen and what is heard impacts perceptual dynamics during binocular rivalry. This bisensory interaction was observed only when the musical score was perceptually dominant, not when it was suppressed from awareness, and it was observed only in people who could read music. Results from two ancillary experiments showed that this effect of congruence cannot be explained by differential patterns of eye movements or by differential response sluggishness associated with congruent score/melody combinations. Taken together, these results demonstrate robust audiovisual interaction based on high-level, symbolic representations and its predictive influence on perceptual dynamics during binocular rivalry.music | binocular rivalry | multisensory interaction | visual awareness | audio-visual congruence V ision can be confusing, particularly when the retinal images inaugurating vision are ambiguous or underspecified with respect to their origins (1, 2). In some situations, confusion can be resolved by reliance on contextual information (3), prior experience (4), expectations (5), and/or implicit knowledge about structural regularities in our visual environment (6). In other situations, however, confusion resists resolution, in which case one can directly experience the dynamical nature of competition as conflicting perceptual interpretations vie for dominance (7). Among the visual phenomena exhibiting this behavior, none is more compelling than binocular rivalry (8).When the two eyes view dissimilar monocular stimuli, stable binocular single vision gives way to interocular competition, characterized by unpredictable fluctuations in visual awareness of the rival stimuli (9, 10). Rivalry offers a way to study the inferential nature of perception when the brain is confronted with conflicting sensory evidence (11,12). Within this context, recent studies imply that rivalry dynamics are governed by influences related to the likelihood of competing perceptual interpretations, such as motor actions (13), affective connotation (14-16), familiarity (17), and concomitant sensory input from other modalities (18-20). Here we examined the limits of susceptibility of rivalry to predictive influences by asking whether binocular rivalry dynamics depend on multisensory congruence between abstract representations familiar to individuals with expertise in that domain of abstraction. The domain we selected was music, for three reasons: (i) musical notation is symbolic; (ii) only people versed in music can decipher that notation; and (iii) melodic structure can be experienced via sight or sound. ResultsExperiment 1: Extended Tracking of Binocular Rivalry. Experiment 1 was implemented as a conventional binocular rivalry task in which participants...

show abstract

Section: Discussionsupporting

confidence: 76%

Section: Discussionmentioning

confidence: 99%

Melodic sound enhances visual awareness of congruent musical notes, but only if you can read music

Lee

Blake

Kim

et al. 2015

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

show abstract

“…Conversely, posterior/central beta power is reduced when goal-dependent scrutiny that imposes constraints on visual processing is minimal and visual processes transpire according to the current biases of the visual system. For example, EEG posterior/central alpha/beta power was reduced while viewing motions that conformed to biological constraints (Meirovitch et al, 2015), consistent with evidence that the visual system is predisposed to processing biological motion (e.g., Allison et al, 2000;Grossman et al, 2000;Plass et al, 2014). Posterior EEG beta power was reduced while experiencing binocular rivalry (Piantoni et al, 2010), that is, when competing percepts alternated according to changes in the adaptive state of the visual system (e.g., Kim et al, 2006;Wilson, 2007;Alais et al, 2010).…”

Section: Introductionsupporting

confidence: 64%

“…In the illusory-motion-reversal displays, the dominant percept of veridical motion occasionally reverses when the motion detectors tuned to the veridical motion sufficiently weaken due to adaptation, that is, when the state of the visual system becomes strongly biased in favor of processing the reversed motion. Evidence suggests that the visual system prioritizes the processing of biologically plausible motions (e.g., Allison et al, 2000;Grossman et al, 2000;Plass et al, 2014), that is, the visual motion mechanisms may be biased in favor of processing motion signals that conform to familiar biological constraints. This interpretation is consistent with the finding that motion-perception related alpha/beta-power reductions were greater for individuals with greater familiarity with the observed movements (Orgs et al, 2008).…”

Section: Discussionmentioning

confidence: 99%

Neural, functional, and aesthetic impacts of spatially heterogeneous flicker: A potential role of natural flicker

Menceloğlu¹,

Grabowecky²,

Suzuki³

2019

Preprint

Self Cite

View full text Add to dashboard Cite

Spatially heterogeneous flicker, characterized by probabilistic and locally independent luminance modulations, abounds in nature. It is generated by flames, water surfaces, rustling leaves, and so on, and it is pleasant to the senses. It affords spatiotemporal multistability that allows sensory activation conforming to the biases of the visual system, thereby generating the perception of spontaneous motion and likely facilitating the calibration of motion detectors. One may thus hypothesize that spatially heterogeneous flicker might potentially provide restoring stimuli to the visual system that engage fluent (requiring minimal top-down control) and selfcalibrating processes. Here, we present some converging behavioral and electrophysiological evidence consistent with this idea. Spatially heterogeneous (multistable) flicker (relative to controls matched in temporal statistics) reduced posterior EEG (electroencephalography) beta power implicated in long-range neural interactions that impose top-down influences on sensory processing. Further, the degree of spatiotemporal multistability, the amount of posterior betapower reduction, and the aesthetic responses to flicker were closely associated. These results are consistent with the idea that the pleasantness of natural flicker may derive from its spatiotemporal multistability that affords fluent and self-calibrating visual processing.

show abstract

“…Recent debates in the literature on multisensory perception have centered on the question of whether audiovisual integration in speech perception occurs automatically or relies on other factors, such as attention (Navarra, Alsius, Soto-Faraco, & Spence, 2010), awareness (Plass, Guzman-Martinez, Ortega, Grabowecky, & Suzuki, 2014), or recognition of speech stimuli as speech related (Tuomainen, Andersen, Tiippana, & Sams, 2005;Vroomen & Stekelenburg, 2011). While visual modulation of auditory speech perception is one of the most studied topics in the field of multisensory perception, far less research has focused on the converse-auditory effects on visual speech perception (Alsius & Munhall, 2013;Baart & Vroomen, 2010;Palmer & Ramsey, 2012;Sweeny et al, 2012).…”

Section: Discussionmentioning

confidence: 99%

Automatic auditory disambiguation of visual awareness

Plass

Guzman-Martinez

Ortega

et al. 2017

Atten Percept Psychophys

Self Cite

View full text Add to dashboard Cite

Multisensory integration can play a critical role in producing unified and reliable perceptual experience. When sensory information in one modality is degraded or ambiguous, information from other senses can crossmodally resolve perceptual ambiguities. Prior research suggests that auditory information can disambiguate the contents of visual awareness by facilitating perception of intermodally consistent stimuli. However, it is unclear whether these effects are truly due to crossmodal facilitation or are mediated by voluntary selective attention to audiovisually congruent stimuli. Here, we demonstrate that sounds can bias competition in binocular rivalry toward audiovisually congruent percepts, even when participants have no recognition of the congruency. When speech sounds were presented in synchrony with speech-like deformations of rivalling ellipses, ellipses with crossmodally congruent deformations were perceptually dominant over those with incongruent deformations. This effect was observed in participants who could not identify the crossmodal congruency in an open-ended interview (Experiment 1) or detect it in a simple 2AFC task (Experiment 2), suggesting that the effect was not due to voluntary selective attention or response bias. These results suggest that sound can automatically disambiguate the contents of visual awareness by facilitating perception of audiovisually congruent stimuli.

show abstract

Lip Reading Without Awareness

Cited by 25 publications

References 15 publications

Melodic sound enhances visual awareness of congruent musical notes, but only if you can read music

Melodic sound enhances visual awareness of congruent musical notes, but only if you can read music

Neural, functional, and aesthetic impacts of spatially heterogeneous flicker: A potential role of natural flicker

Automatic auditory disambiguation of visual awareness

Contact Info

Product

Resources

About