Visual speech cues enhance neural speech tracking in right auditory cluster leading to improvement in speech in noise comprehension in older adults with hearing impairment

Abstract

Comprehending speech in noisy situations is difficult and suffers with aging, hearing loss, and cognitive decline. Using visual speech cues improves perception and comprehension of speech in noise (SiN) in older adults with hearing impairment. In the current study we investigated neurofunctional correlates (i.e., neural tracking of speech) of the visual enhancement effect in a naturalistic setting in which older adults with hearing impairment (N=67) listened to sentences in babble noise in audio-only and audiovisual conditions and performed an intelligibility and a comprehension task, while EEG was recorded. Behaviorally, we found that visual speech cues improved intelligibility and comprehension of speech presented in babble noise and that particularly individuals with high working memory capacity benefitted. Furthermore, we found that the visual enhancement effect was accompanied by increased cortical tracking of the speech envelope particularly in individuals who were low performers in the audio-only condition (i.e., < 70% correct) leading to improved speech in noise comprehension in an audiovisual setting. Further, these low auditory-only performers showed deficient neural speech tracking in a right auditory topographical cluster, which improved significantly when visual speech cues were presented leading to more bilateral processing of speech. Overall, our results show that older adults with hearing impairment who have difficulties with speech in noise perception use visual speech cues to improve performance which is reflected in more bilateral processing of speech.

Presenting natural continuous speech in a multisensory immersive environment improves speech comprehension and reflects the allocation of processing resources in neural speech tracking.

“…The study included 40 healthy retired participants (M age = 70, Range Age = 61-80, SD Age = 4.92, male = 20) who did not report pre-existing neurological or psychiatric conditions or any speech or language disorders. Sample size was based on previous work investigating behavioural and neural correlates of multimodal speech perception within the context of cognitive load (Frei et al, 2023;Frtusova & Phillips, 2016;Hjortkjaer et al, 2020;Schmitt et al, 2022). For the electrophysiological analysis three participants had to be excluded, due to constant noise over one of the two measurement sessions, leaving a total of 37 participants in the study.…”

Perceptual Doping: A Hypothesis on How Early Audiovisual Speech Stimulation Enhances Subsequent Auditory Speech Processing

“…Further, Mégevand et al [ 19 ], in a study of the electrical activity of the human brain via implanted electrodes, found that visual speech cues in an audiovisual speech signal improved phase-tracking and reduced the amplitude of evoked responses to congruent auditory speech signals. Frei et al [ 20 ] also reported that visual speech cues increase neural tracking of the speech cues, particularly in the right auditory clusters, which subsequently results in better speech in noise comprehension in older adults with hearing loss. We speculate that after audiovisual speech stimulation, the improved capacity of the auditory cortex to track down the most critical features of speech, particularly in degraded listening conditions, does not vanish and is expected to persist for a longer period.…”