Decoding semantic predictions from EEG prior to word onset

Abstract

The outstanding speed of language comprehension necessitates a highly efficient implementation of cognitive-linguistic processes. The domain-general theory of Predictive Coding suggests that our brain solves this problem by continuously forming linguistic predictions about expected upcoming input. The neurophysiological implementation of these predictive linguistic processes, however, is not yet understood. Here, we use EEG (human participants, both sexes) to investigate the existence and nature of online-generated, categorylevel semantic representations during sentence processing. We conducted two experiments in which some nouns -embedded in a predictive spoken sentence context -were unexpectedly delayed by 1 second. Target nouns were either abstract/concrete (Experiment 1) or animate/inanimate (Experiment 2). We hypothesized that if neural prediction error signals following (temporary) omissions carry specific information about the stimulus, the semantic category of the upcoming target word is encoded in brain activity prior to its presentation. Using time-generalized multivariate pattern analysis, we demonstrate significant decoding of word category from silent periods directly preceding the target word, in both experiments. This provides direct evidence for predictive coding during sentence processing, i.e., that information about a word can be encoded in brain activity before it is perceived. While the same semantic contrast could also be decoded from EEG activity elicited by isolated words (Experiment 1), the identified neural patterns did not generalize to pre-stimulus delay period activity in sentences. Our results not only indicate that the brain processes language predictively, but also demonstrate the nature and sentence-specificity of category-level semantic predictions preactivated during sentence comprehension.Keywords: predictive coding, sentence processing, semantics, delayed targets, EEG, multivariate pattern analysis . CC-BY-NC-ND 4.0 International license It is made available under a (which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.The copyright holder for this preprint . http://dx.doi.org/10.1101/393066 doi: bioRxiv preprint first posted online Aug. 16, 2018; 3 STATEMENT OF SIGNIFICANCE:The speed of language comprehension necessitates a highly efficient implementation of cognitive-linguistic processes. Predictive processing has been suggested as a solution to this problem, but the underlying neural mechanisms and linguistic content of such predictions are only poorly understood. Inspired by Predictive Coding theory, we investigate whether the meaning of expected, but not-yet heard words can be decoded from brain activity. Using EEG, we can predict if a word is, e.g., abstract (as opposed to concrete), or animate (vs. inanimate), from brain signals preceding the word itself. This strengthens predictive coding theory as a likely candidate for the principled neural mechanisms underlying online processing of lan...

Predictive pre‐activation of orthographic and lexical‐semantic representations facilitates visual word recognition

“…First studies showed that language‐related brain regions can be activated before a highly expected stimulus appears (Bonhage et al., 2015; Dikker & Pylkkanen, 2011; Wang et al., 2017), but did not further assess the nature of pre‐activated representations. More recent work showed pre‐stimulus effects of semantic category (Heikel et al., 2018; Wang et al., 2020) using multivariate pattern analysis techniques (King et al., 2018; Kragel et al., 2018), as well as pre‐stimulus effects of word frequency (Fruchter et al., 2015) using linear mixed models (LMMs; Baayen et al., 2008). These studies provide initial evidence that lexical‐semantic representations are pre‐activated before a predictable word appears.…”

Predictive pre-activation of orthographic and lexical-semantic representations facilitates visual word recognition

“…We had initially favored multivariate pattern analysis (MVPA) as method of choice for the investigation of pre-activation effects in the MEG data, as previous studies had successfully decoded linguistic information from M/EEG data even in the absence of external stimulation (Simanova et al, 2015;Heikel et al, 2018). However, with this procedure we observed that the statistical power was limited and only resulted in significant findings for the strong and categorical lexicality effect during prime processing (Fig.…”

“…Results 5), which has been established in previous work for investigating brain responses in the absence of external stimulation (e.g.,Simanova et al, 2015;Heikel et al, 2018), a power analysis indicated higher power and more reliable effect size estimates for LMMs in contrast to multivariate pattern decoding (see Supporting Results 6). Importantly, LMMs can estimate the effects of interest while controlling for confounding variables.…”

“…presented in Figure 3, as well as of the largest overlapping clusters thresholded at 50 % of the overall peak) across The difference between prime and target did not reach significance for any contrast or time point (Bonferronicorrected). We had initially favored the MVPA analysis as method of choice for the investigation of pre-activation effects in the MEG data, as previous studies had successfully decoded linguistic information from M/EEG data even in the absence of external stimulation (Simanova et al, 2015;Heikel et al, 2018;). However, with this procedure we hardly found significant results for the letter string characteristics indicated as relevant in the behavioral analysis, with the exception of lexicality effects during prime processing.…”

“…First studies showed that language-related brain regions can be activated before a highly expected stimulus appears (Dikker and Pylkkanen, 2011;Bonhage et al, 2015;Wang et al, 2017), but did not further assess the nature of pre-activated representations. More recent work showed pre-stimulus effects of semantic category (Heikel et al, 2018;Wang et al, 2020) using multivariate pattern analysis techniques (Kragel et al, 2018;King et al, 2020), as well as pre-stimulus effects of word frequency (Fruchter et al, 2015) using linear mixed models (LMMs;Baayen et al, 2008). These studies provide initial evidence that lexical-semantic representations are pre-activated before a predictable word appears.…”