Visual speech differentially modulates beta, theta, and high gamma bands in auditory cortex

Abstract

Speech perception is a central component of social communication. While speech perception is primarily driven by sounds, accurate perception in everyday settings is also supported by meaningful information extracted from visual cues (e.g., speech content, timing, and speaker identity). Previous research has shown that visual speech modulates activity in cortical areas subserving auditory speech perception, including the superior temporal gyrus (STG), likely through feedback connections from the multisensory posterior superior temporal sulcus (pSTS). However, it is unknown whether visual modulation of auditory processing in the STG is a unitary phenomenon or, rather, consists of multiple temporally, spatially, or functionally discrete processes. To explore these questions, we examined neural responses to audiovisual speech in electrodes implanted intracranially in the temporal cortex of 21 patients undergoing clinical monitoring for epilepsy. We found that visual speech modulates auditory processes in the STG in multiple ways, eliciting temporally and spatially distinct patterns of activity that differ across theta, beta, and high-gamma frequency bands. Before speech onset, visual information increased high-gamma power in the posterior STG and suppressed beta power in mid-STG regions, suggesting crossmodal prediction of speech signals in these areas. After sound onset, visual speech decreased theta power in the middle and posterior STG, potentially reflecting a decrease in sustained feedforward auditory activity. These results are consistent with models that posit multiple distinct mechanisms supporting audiovisual speech perception.

The role of informational content of visual speech in an audiovisual cocktail party: Evidence from cortical oscillations in young and old participants

“…Oscillations in the beta band (16 to 30 Hz) have been shown to be closely intertwined with alpha oscillations during multisensory tasks requiring attentional inhibition (Friese et al, 2016; Ganesan et al, 2021). Beta oscillations are associated with neural networks responsible for error‐monitoring and decision making (Arnal & Giraud, 2012; Friese et al, 2016) and the maintenance of current sensorimotor information (Engel & Fries, 2010; Schneider et al, 2020).…”

Effect of visual input on syllable parsing in a computational model of a neural microcircuit for speech processing

“…Experimental data could compare the power spectral density of EEG waves and the firing rates to see how they correspond to the simulations on the spectra of the LFP that we have done here to further shed light on the multisensory mechanism of audiovisual processing in the brain. Further studies could also tell us how visual speech affects the different oscillatory bands spatiotemporally across the auditory cortex and compare the results with the experimental data [65]. Integrating such neural data with behavioural data on speech comprehension in a computational model will further clarify the neural mechanisms of audiovisual speech processing.…”

Age influences audiovisual speech processing in multi-talker scenarios – Evidence from cortical oscillations

“…Oscillations in the beta band (16 to 30 Hz) have been shown to be closely intertwined with alpha oscillations during tasks requiring attentional inhibition in multisensory tasks (Friese et al, 2016; Ganesan et al, 2021). Beta oscillations are associated with neural networks responsible for error-monitoring and decision making (Arnal & Giraud, 2012; Friese et al, 2016) and the maintenance of current sensorimotor information (Engel & Fries, 2010; Schneider et al, 2020).…”