Visual inputs decrease brain activity in frontal areas during silent lipreading

Bello, Julio Plata; García-Peña, Carlos; Modroño, Cristián; Hernández‐Martín, Estefanía; Pérez-Martín, Yaiza; Marcano, Francisco; González–Mora, José Luis

doi:10.1371/journal.pone.0223782

Cited by 4 publications

(3 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Visemes similarly modulated activity broadly through the STG, with increased BOLD at the posterior STG and decreased BOLD in the middle to anterior STG, along with increased BOLD in visual regions. The finding that lipreading suppresses neural activity within portions of the auditory system is consistent with prior reports from fMRI 24 and iEEG 25 , which has been theorized to reflect the optimized tuning of neurons specialized for auditory speech 26 .…”

Section: Resultssupporting

confidence: 89%

Auditory cortex encodes lipreading information through spatially distributed activity

Ganesan

Cao

Demidenko

et al. 2022

Preprint

View full text Add to dashboard Cite

Face-to-face communication improves the quality and accuracy of heard speech, particularly in noisy environments. Silent lipreading modulates activity in auditory regions, which has been hypothesized to reflect the transformation and encoding of multiple forms of visual speech information used to support hearing processes. Evidence suggests visual timing information as one such signal encoded in auditory areas: seeing when a speaker's lips come together between words can help listeners parse word-level boundaries. However, it remains unclear how lipreading alters activity in the auditory system to improve speech perception at the single word-level. Using fMRI and intracranial electrodes in patients, here we show that silently lipread words can be classified from neural activity in auditory areas based on distributed spatial information. Lipread words evoked similar representations to the corresponding heard words, consistent with the prediction that automatic lipreading refines the tuning of auditory representations. Similar to heard words, lipread words varied in the distinctiveness of their neural representations in auditory cortex: e.g., the lipread words DIG and GIG evoked more similar neural activity in auditory cortex relative to the more perceptually distinct word FIG, suggesting that lipreading activity reflects probabilistic distributions as opposed to the unique identity of the lipread word. Notably, while visual speech has both excitatory and suppressive effects on auditory firing rates, classification was observed in both neural populations, consistent with the prediction that lipreading contributes to phoneme population tuning by both activating the corresponding representation and suppressing incorrect phonemic representations. These results support a model in which the auditory system combines the joint neural distributions evoked by heard and lipread words to generate a more precise estimate of what was said, particularly during noisy speech.

show abstract

Section: Resultssupporting

confidence: 89%

Auditory cortex encodes lipreading information through spatially distributed activity

Ganesan

Cao

Demidenko

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Furthermore, the SMA is crucial for sensorimotor integration involved in generating actions (Ruan et al, 2018 ). Notably, a recent fMRI study revealed that, during silent lipreading, brain activity in the SMA decreased when an additional visual stimulus was included (Plata Bello et al, 2019 ). Consistent with these findings, we observed that patients with TAO had decreased interhemispheric FC within the sensorimotor network, which suggests impaired sensorimotor function in these patients.…”

Section: Discussionmentioning

confidence: 99%

Reduction of Interhemispheric Homotopic Connectivity in Cognitive and Visual Information Processing Pathways in Patients With Thyroid-Associated Ophthalmopathy

Wen

Huang

2022

Front. Hum. Neurosci.

View full text Add to dashboard Cite

PurposeThyroid-associated ophthalmopathy (TAO) is a vision threatening autoimmune and inflammatory orbital disease, and has been reported to be associated with a wide range of structural and functional abnormalities of bilateral hemispheres. However, whether the interhemisphere functional connectivity (FC) of TAO patients is altered still remain unclear. A new technique called voxel-mirrored homotopic connectivity (VMHC) combined with support vector machine (SVM) method was used in the present study to explore interhemispheric homotopic functional connectivity alterations in patients with TAO.MethodsA total of 21 TAO patients (14 males and 7 females) and 21 wellmatched healthy controls (HCs, 14 males and 7 females), respectively, underwent functional magnetic resonance imaging (fMRI) scanning in the resting state. We evaluated alterations in the resting state functional connectivity between hemispheres by applying VMHC method and then selected these abnormal brain regions as seed areas for subsequent study using FC method. Furthermore, the observed changes of regions in the VMHC analysis were chosen as classification features to differentiate patients with TAO from HCs through support vector machine (SVM) method.ResultsThe results showed that compared with HCs, TAO patients showed significantly lower VMHC values in the bilateral postcentral gyrus, lingual gyrus, calcarine, middle temporal gyrus, middle occipital gyrus and angular. Moreover, significantly decreased FC values were found between the right postcentral gyrus/lingual gyrus/calcarine and left lingual gyrus/cuneus/superior occipital gyrus, left postcentral gyrus/lingual gyrus/calcarine and right lingual gyrus/ middle temporal gyrus, right middle temporal gyrus and left cerebellum-8/lingual gyrus/middle occipital gyrus/supplementary motor area, left middle temporal gyrus and right middle occipital gyrus, right middle occipital gyrus/angular and left middle temporal pole (voxel-level p < 0.01, Gaussian random field correction, cluster-level p < 0.05). The SVM classification model achieved good performance in differentiating TAO patients from HCs (total accuracy: 73.81%; area under the curve: 0.79).ConclusionThe present study revealed that the altered interhemisphere interaction and integration of information involved in cognitive and visual information processing pathways including the postcentral gyrus, cuneus, cerebellum, angular, widespread visual cortex and temporal cortex in patients with TAO relative to HC group. VMHC variability had potential value for accurately and specifically distinguishing patients with TAO from HCs. The new findings may provide novel insights into the neurological mechanisms underlying visual and cognitive disorders in patients with TAO.

show abstract

“…The McGurk paradigm consists of the concurrent presentation of either congruent or incongruent auditory and visual signals [ 14 ]. Interpretations of Jenson et al [ 30 ] notwithstanding, the capacity for lip-reading [ 45 ] and visual speech identification [ 46 , 47 ] may be interpreted to suggest that the initial articulatory hypotheses supporting Analysis by Synthesis may be generated by either auditory or visual signals. It may then be proposed that in the McGurk paradigm, both visual and auditory streams give rise to independent articulatory hypotheses, which must be reconciled to allow the extraction of a unified phonological representation during working memory encoding.…”

Section: Introductionmentioning

confidence: 99%

Audiovisual incongruence differentially impacts left and right hemisphere sensorimotor oscillations: Potential applications to production

Jenson

2021

PLoS ONE

View full text Add to dashboard Cite

Speech production gives rise to distinct auditory and somatosensory feedback signals which are dynamically integrated to enable online monitoring and error correction, though it remains unclear how the sensorimotor system supports the integration of these multimodal signals. Capitalizing on the parity of sensorimotor processes supporting perception and production, the current study employed the McGurk paradigm to induce multimodal sensory congruence/incongruence. EEG data from a cohort of 39 typical speakers were decomposed with independent component analysis to identify bilateral mu rhythms; indices of sensorimotor activity. Subsequent time-frequency analyses revealed bilateral patterns of event related desynchronization (ERD) across alpha and beta frequency ranges over the time course of perceptual events. Right mu activity was characterized by reduced ERD during all cases of audiovisual incongruence, while left mu activity was attenuated and protracted in McGurk trials eliciting sensory fusion. Results were interpreted to suggest distinct hemispheric contributions, with right hemisphere mu activity supporting a coarse incongruence detection process and left hemisphere mu activity reflecting a more granular level of analysis including phonological identification and incongruence resolution. Findings are also considered in regard to incongruence detection and resolution processes during production.

show abstract

Visual inputs decrease brain activity in frontal areas during silent lipreading

Cited by 4 publications

References 31 publications

Auditory cortex encodes lipreading information through spatially distributed activity

Auditory cortex encodes lipreading information through spatially distributed activity

Reduction of Interhemispheric Homotopic Connectivity in Cognitive and Visual Information Processing Pathways in Patients With Thyroid-Associated Ophthalmopathy

Audiovisual incongruence differentially impacts left and right hemisphere sensorimotor oscillations: Potential applications to production

Contact Info

Product

Resources

About