The role of informational content of visual speech in an audiovisual cocktail party: Evidence from cortical oscillations in young and old participants

Abstract

Age-related differences in the processing of audiovisual speech in a multitalker environment were investigated analysing event-related spectral perturbations (ERSPs), focusing on theta, alpha and beta oscillations that are assumed to reflect conflict processing, multisensory integration and attentional mechanisms, respectively. Eighteen older and 21 younger healthy adults completed a two-alternative forced-choice word discrimination task, responding to audiovisual speech stimuli. In a cocktail-party scenario with two competing talkers (located at À15 and 15 azimuth), target words (/yes/or/no/) appeared at a pre-defined (attended) position, distractor words at the other position. In two audiovisual conditions, acoustic speech was combined either with informative or uninformative visual speech. While a behavioural benefit for informative visual speech occurred for both age groups, differences between audiovisual conditions in the theta and beta band were only present for older adults. A stronger increase in theta perturbations for stimuli containing uninformative visual speech could be associated with early conflict processing, while a stronger suppression in beta perturbations for informative visual speech could be associated to audiovisual integration. Compared to the younger group, the older group showed generally stronger beta perturbations. No condition differences in the alpha band were found. Overall, the findings suggest age-related differences in audiovisual speech integration in a multi-talker environment. While the behavioural benefit of informative visual speech was unaffected by age, older adults had a stronger need for cognitive control when processing conflicting audiovisual speech input. Furthermore, mechanisms of

Cross-modal interactions at the audiovisual cocktail-party revealed by behavior, ERPs, and neural oscillations

“…Participants' performance was similar to the performance of young adults in two previous studies, using the same test procedure (Begau et al 2021(Begau et al , 2022.…”

“…To increase the power to detect a true effect, we constrained the electrode sites and time points included in the analysis. That is, based on our previous study on audiovisual speech processing using similar conditions and stimulus material (Begau et al 2022), we averaged the data across a cluster of fronto-central channels (including Cz, Fz, C1, C2). In addition, as we were not interested in anticipatory effects, only post-stimulus time points (i.e., in-between 0 and 1782 ms) were considered for the statistical analysis.…”

“…Previous studies highlight the importance of synchronization of neural oscillations for multisensory processing (for a review, see Keil & Senkowski, 2018). Our own recent work, employing a similar audiovisual cocktail-party paradigm, suggests that in particular oscillatory power in the theta frequency over fronto-central scalp sites is modulated by the informational content of visual speech (Begau et al, 2022). Therefore, we performed an exploratory time-frequency analysis, using a cluster-based permutation analysis (see methods section for details).…”

“…The acoustic speech stimuli were either presented (a) without any visual information (auditory only), (b) together with congruent visual information (i.e., a recording of a speaker's face, in which the lip movement matched the presented auditory information) or (c) together with uninformative visual information (i.e., a recording of a speaker's face, in which the lip movement did not match, but pronouncing a 'ba' lip movement). Noteworthy, as in previous investigations (Begau et al, 2021(Begau et al, , 2022, the visual stimulus material consisted of natural recordings of a speaker's face, allowing for an ecologically valid simulation of a multi-speaker scenario. This allowed us to answer three overarching research questions: First, we aimed to investigate to what extend the presence of congruent or uninformative visual information aids the listener to select the relevant sound from a mixture of competing inputs in a multitalker environment.…”

“…Moreover, a growing body of work emphasizes the relevance of neural oscillations for multisensory processing (reviewed by Keil & Senkowski, 2018). In particular, modulations of theta power (4-7 Hz) have been associated with the detection of audiovisual incongruency and the need for cognitive control during audiovisual integration (Begau et al, 2022; Morís Fernández et al, 2018). Hence, in addition, time-frequency dynamics underlying attentional orienting in this multisensory cocktail-party scenario were explored.…”