Auditory detection is modulated by theta phase of silent lip movements

Abstract

<jats:title>SUMMARY</jats:title><jats:p>Audiovisual speech perception relies on our expertise to map a speaker’s lip movements with speech sounds. This multimodal matching is facilitated by salient syllable features that align lip movements and acoustic envelope signals in the 4 – 8 Hz theta band (Chandrasekaran et al., 2009). The predominance of theta rhythms in speech processing has been firmly established by studies showing that neural oscillations track the acoustic envelope in the primary auditory cortex (Giraud &amp; Poeppel, 2012). Equivalently, theta oscillations in the visual cortex entrain to lip movements (Park et al., 2016), and the auditory cortex is recruited during silent speech perception (Bourguignon et al., 2020; Cross et al., 2019; Calvert et al., 1997). These findings suggest that neuronal theta oscillations play a functional role in organising information flow across visual and auditory sensory areas. We presented silent speech movies while participants performed a pure tone detection task to test whether entrainment to lip movements enslaves the auditory system and drives behavioural outcomes. We showed that auditory detection varied depending on the ongoing theta phase conveyed by lip movements in the movies. In a complementary experiment presenting the same movies while recording participants’ electro-encephalogram (EEG), we found that silent lip movements entrained neural oscillations in the visual and auditory cortices with the visual phase leading the auditory phase. These results support the idea that the visual cortex entrained by lip movements increases the sensitivity of the auditory cortex at relevant time-windows for speech comprehension as a filtering modulator relying on theta phase synchronisation.</jats:p>

Recurrent neural network model of human event-related potentials in response to intensity oddball stimulation

“…Furthermore, other sources of interference with putative MMN measurements may emerge as unintended consequences of subject behaviour during passive auditory oddball experiments. For instance, two common approaches are to instruct the subject to watch a silent film or read a book, which present different sets of cues that could unintentionally modulate the auditory response in different respects (Biau et al, 2021; Rayner and Clifton, 2009; Zoefel, 2021). As such, demonstrating that MMN reflects a specific prediction-error signal is challenging, and the balance of evidence from the current study leans decidedly towards the hypothesis that intensity MMN reflects modulation of obligatory sensory components of the ERP.…”

Oscillation or not—Why we can and need to know (commentary on Doelling and Assaneo, 2021)

“…We illustrate our line of argumentation with an example. Recent research has shown that the rhythm of visual speech cues modulates the ability to detect an auditory target (Biau et al, 2021). This effect could merely reflect a passive relay of visual information to the auditory system and lead to a rhythmic modulation of auditory perception only because the information itself is rhythmic.…”

Oscillation or not – why we can and need to know

“…We illustrate our line of argumentation with an example. Recent research has shown that the rhythm of visual speech cues modulates the ability to detect an auditory target [16]. This effect could merely reflect a passive relay of visual information to the auditory system and lead to a rhythmic modulation of auditory perception only because the information itself is rhythmic.…”