Get the gist of the story: Neural map of topic keywords in multi-speaker environment

Abstract

Neural representation of lexico-semantics in speech processing has been revealed in recent years. However, to date, how the brain makes sense of the higher-level semantic gist (topic keywords) of a continuous speech remains mysterious. Capitalizing on a generative probabilistic topic modelling algorithm on speech materials to which participants listened while their brain activities were recorded by Magnetoencephalography (MEG), here we show spatio-temporal neural representation of topic keywords in a multi-speaker environment where task-relevant (attended) and -irrelevant (unattended) speech co-exits. We report the difference of neural representation between salient and less salient semantic gist of both attended and unattended speech. Moreover, we show that greater sensitivity to semantically salient unattended speech in the left auditory and motor cortices negatively mediates attended speech comprehension.

Differential roles of delta and theta oscillations in understanding semantic gist during natural audiovisual speech perception: Functional and anatomical evidence

“…Visual speech (lip movement) was matched to the auditory speech to which the participants paid attention. LDA topic model applied speech chunks were split in to high and low topic probability conditions according to the highest topic cluster (see Park and Gross (2022), which represents the main idea of the talk. MEG data, auditory speech envelope and visual speech signal were band-pass filtered to delta (1-3 Hz) and theta (4-7 Hz) bands, and the DICS source filters were applied to the MEG data.…”

“…Our previous report demonstrated brain activities involved in the understanding of topic keywords during natural speech perception using a computational topic modelling algorithm in combination with the encoding and decoding model approach (Park and Gross, 2022). Speech chunks were first derived based on acoustic properties using silence threshold in continuous speech, which can be considered as perceptual chunks above phrasal level.…”

“…In order to apply the topic model (see below) to the current speech materials, we first segmented speech text from each talk into multiple chunks based on their acoustic properties. Using the Syllable Nuclei (de Jong and Wempe, 2009) in Praat software (Boersma and Weenink, 2018), we obtained acoustic chunks with parameters of 0.25 s minimum pause (silence) durations, -25 dB silence threshold, and 1 s minimum duration of each speech chunk (Figure 1a in Park and Gross (2022)). The segmentation resulted in 129 speech chunks on average with a mean duration of 3.37 s from the seven talks used for auditory speech talks.…”

“…The maximum values for positive, neutral and negative are 1.0, and the compound score is a normalized value across positive, neutral, and negative scores to be between -1 and 1. The results for talks show near-zero compound scores meaning that the speech materials are all with a neutral sentiment (see Supplementary Table 2 in Park and Gross (2022).…”

“…However, the role of low-frequency brain rhythms underlying how our brain extracts high-level semantic gist during naturalistic audiovisual speech perception remains largely unknown. In our previous study, we have demonstrated that the neural mechanisms underlying higher-order semantic systems above word-level, such as topic processing during natural speech perception, can be mapped (Park and Gross, 2022).…”