Neural tracking measures of speech intelligibility: Manipulating intelligibility while keeping acoustics unchanged

Abstract

Neural speech tracking has advanced our understanding of how our brains rapidly map an acoustic speech signal onto linguistic representations and ultimately meaning. It remains unclear, however, how speech intelligibility is related to the corresponding neural responses. Many studies addressing this question vary the level of intelligibility by manipulating the acoustic waveform, but this makes it difficult to cleanly disentangle the effects of intelligibility from underlying acoustical confounds. Here, using magnetoencephalography recordings, we study neural measures of speech intelligibility by manipulating intelligibility while keeping the acoustics strictly unchanged. Acoustically identical degraded speech stimuli (three-band noise-vocoded, ~20 s duration) are presented twice, but the second presentation is preceded by the original (nondegraded) version of the speech. This intermediate priming, which generates a “pop-out” percept, substantially improves the intelligibility of the second degraded speech passage. We investigate how intelligibility and acoustical structure affect acoustic and linguistic neural representations using multivariate temporal response functions (mTRFs). As expected, behavioral results confirm that perceived speech clarity is improved by priming. mTRFs analysis reveals that auditory (speech envelope and envelope onset) neural representations are not affected by priming but only by the acoustics of the stimuli (bottom–up driven). Critically, our findings suggest that segmentation of sounds into words emerges with better speech intelligibility, and most strongly at the later (~400 ms latency) word processing stage, in prefrontal cortex, in line with engagement of top–down mechanisms associated with priming. Taken together, our results show that word representations may provide some objective measures of speech comprehension.

Neural decoding of the speech envelope: Effects of intelligibility and spectral degradation

“…We, moreover, quantify the generalisability of speech decoding in the context of our experimental manipulations, as well as across subject-specific versus group-level reconstruction techniques. Our findings cohere with an emerging consensus that envelope tracking is possible without speech understanding (Baltzell et al, 2017; Gillis et al, 2023; Karunathilake et al, 2023; Kösem et al, 2023), yet intelligibility nonetheless exerts an enhancing effect on decoding accuracy in the current dataset and thereby supports our first hypothesis.…”

“…In particular, Xu and colleagues (2023) argued that the sEEG data reveal a functional dissociation between automatic processing of low-level acoustic information, and a slightly later response that is selective for intelligible speech; however, as speech intelligibility was fully confounded with acoustic clarity in their study, it is unknown whether spatial-temporal changes in the neural response are the result of waning comprehension or increasing spectral degradation. A shift in perceptual attention from one putative system to the other could, in theory, contribute to the sudden “pop-out” listeners sometimes experience when hearing vocoded speech (Corcoran et al, 2023; Davis et al, 2005; Karunathilake et al, 2023; Kösem et al, 2023). Presumably, under ecologically relevant conditions, both the automatic acoustic and speech-selective systems would continuously interact at multiple stages of auditory processing.…”

Speech-reception-threshold estimation via EEG-based continuous speech envelope reconstruction

“…However, significant differences in P1 and N1 amplitudes were found when comparing vocoded and non-vocoded speech, indicating that the changes we observed in TRFs may reflect envelope representation rather than a direct measure of speech intelligibility. In the Karunathilake et al (2023) study, TRFs obtained from an encoder using word onsets showed two deflections, an early positive deflection (~100 ms) and a later negative deflection (~400 ms), both modulated by speech intelligibility, with a greater effect size observed for the later deflection. Further research is needed to determine if using word onsets instead of the speech envelope would provide a more direct measure of speech intelligibility.…”

“…It has also been shown that the reconstruction accuracy of the envelope is correlated with speech intelligibility (Ding and Simon, 2013; Iotzov and Parra, 2019; Vanthornhout et al, 2018) and that the SRT beh can be predicted from the reconstruction accuracy of the envelope (Lesenfants et al, 2019). A recent study (Karunathilake et al, 2023) suggests that the reconstruction accuracy of the envelope is not a direct speech intelligibility measure. They used stimuli of vocoded speech and manipulate speech intelligibility by either introducing the clean speech version of the vocoded speech before or not, thus manipulating the intelligibility of the vocoded speech.…”

“…In the study by Karunathilake et al (2023), it was observed that there was no significant difference in TRF amplitudes when manipulating only the intelligibility while keeping the acoustics of the stimuli constant, using an encoder with envelope as a stimuli feature. However, significant differences in P1 and N1 amplitudes were found when comparing vocoded and non-vocoded speech, indicating that the changes we observed in TRFs may reflect envelope representation rather than a direct measure of speech intelligibility.…”