Dynamic EEG analysis during language comprehension reveals interactive cascades between perceptual processing and sentential expectations

Abstract

Understanding spoken language requires analysis of the rapidly unfolding speech signal at multiple levels: acoustic, phonological, and semantic. However, there is not yet a comprehensive picture of how these levels relate. We recorded electroencephalography (EEG) while listeners (N=31) heard sentences in which we manipulated acoustic ambiguity (e.g., a bees/peas continuum) and sentential expectations (e.g., Honey is made by bees). EEG was analyzed with a mixed effects model over time to quantify how language processing cascades proceed on a millisecond-by-millisecond basis. Our results indicate: (1) perceptual processing and memory for fine-grained acoustics is preserved in brain activity for up to 900 msec; (2) contextual analysis begins early and is graded with respect to the acoustic signal; and (3) top-down predictions influence perceptual processing in some cases, however, these predictions are available simultaneously with the veridical signal. These mechanistic insights provide a basis for a better understanding of the cortical language network.

The time‐course of speech perception revealed by temporally‐sensitive neural measures

“…Moreover, this response did not depend on the listeners' ultimate behavioral categorization of the sound, suggesting that the N1 response tracks the VOT of the stimulus independently of phonological representations. Effects of N1 amplitude reflecting gradient encoding of VOT have also been observed in other studies (Noe & Fischer‐Baum, 2020; Sarrett, McMurray, & Kapnoula, submitted). These results suggest that the auditory N1 serves as an index of early cue encoding for speech sounds.…”

“…ECoG data from experiments using nonspeech materials also reveal considerable sensitivity to graded acoustic differences (Nourski et al, 2014, 2015). Thus, although some ECoG studies suggest evidence for responses consistent with categorical perception in pSTG, others suggest that this area maintains sensitivity to subphonemic information, consistent with work using scalp‐recorded EEG (Getz & Toscano, 2019; Noe & Fischer‐Baum, in press; Sarrett et al, submitted; Toscano et al, 2010). It may also be the case that both types of representations are maintained in STG (Yi, Leonard, & Chang, 2019).…”

“…Other work also suggests that gradient sensitivity persists over extended periods of time. For example, Sarrett et al (submitted) found evidence for graded VOT difference as late as 700 ms after stimulus onset. Similarly, work from the visual‐world eye‐tracking paradigm demonstrates that gradiency persists over the course of a single word (McMurray et al, 2009) and over multi‐word utterances (Brown‐Schmidt & Toscano, 2017; Falandays et al, 2020; Zellou & Dahan, 2019).…”

“…Data from MEG experiments provide further support for parallel processing of cue‐ and category‐level information (Gwilliams, Linzen, Poeppel, & Marantz, 2018). Thus, gradient sensitivity is maintained even while other, higher‐level aspects of spoken language are processed (e.g., phonological categories, semantic information; Sarrett et al, submitted; Toscano et al, 2018). Additional work is needed to determine whether the same or different cortical areas mediate these responses.…”

“…In a similar experiment, Sarrett et al (submitted) investigated how semantic expectations in more dynamic sentential contexts interact with the early time‐course of speech perception. Listeners performed a phoneme identification task in which they heard sentences that biased them to expect a sentence‐final target word beginning with either /b/ or /p/ (e.g., “The outer part of a tree is called bark ” vs. “There are several roller coasters in that amusement park ”) with stimuli varying along VOT continua.…”