Graded sensitivity to structure and meaning throughout the human language network

Abstract

How are syntactically and semantically connected word sequences, or constituents, represented in the human language system? An influential fMRI study, Pallier et al. (2011, PNAS), manipulated the length of constituents in sequences of words or pseudowords. They reported that some language regions (in the anterior temporal cortex and near the temporo-parietal junction) were sensitive to constituent length only for sequences of real words but not pseudowords. In contrast, language regions in the inferior frontal and posterior temporal cortex showed the same pattern of increased response to longer constituents—and similar overall response magnitudes— for word and pseudoword sequences. Based on these results, Pallier et al. argued that the latter regions represent abstract sentence structure. Here we identify methodological and theoretical concerns with the Pallier et al. study and conduct a replication across two fMRI experiments. Our results do not support Pallier et al.’s critical claim of distinct neural specialization for abstract syntactic representations. Instead, we find that all language regions show a similar profile of sensitivity to both constituent length and lexicality (stronger responses to real-word than pseudoword stimuli). In addition, we argue that the constituent length effect in these experiments i) is not readily grounded in established theories of sentence processing, and ii) may not actually derive from syntactic structure building, but may instead reflect the temporal receptive window of the human language system.Significance StatementAn influential study (Pallier et al., 2011, PNAS) has been widely interpreted as evidence that inferior frontal and posterior temporal brain regions perform abstract syntactic processing during language comprehension, independent of lexical content. Here we identify theoretical, empirical, and methodological concerns with the original study and conduct a replication across two experiments. Contrary to Pallier et al. (2011), we find evidence that both syntactic and lexical processing are distributed throughout the human language network, consistent with prior arguments for a highly lexicalized and distributed human language processing system.

Long-range and hierarchical language predictions in brains and algorithms

“…Finally, we use a recent method to decompose these neural activations into syntactic and semantic representations (33), and show that the longrange forecasts are predominantly driven by semantic features. This finding strengthens the idea that while syntax may be explicitly represented in neural activity (52,53), predicting high-level meaning may be at the core of language processing (54,55).…”

The language network reliably ‘tracks’ naturalistic meaningful non-verbal stimuli

“…One constraint in this space of hypotheses has to do with the size of the language network's 'temporal integration (or receptive) window' (e.g., Lerner et al, 2011). In particular, previous work has shown that the temporal integration window of the language network is relatively short, on the order of a clause or sentence (e.g., Lerner et al, 2011;Blank & Fedorenko, 2020;Shain et al, 2021). It is therefore likely that non-verbal meanings that the language system is concerned with have to do with events, but not longertimescale representations like situation models whose construction can span long sequences of events (Johnson-Laird, 1983;Zwaan & Radvansky, 1998;Loschky et al, 2019).…”

Precision fMRI reveals that the language-selective network supports both phrase-structure building and lexical access during language production

“…However, an architecture where different kinds of syntactic dependencies are supported by distinct mechanisms seems unlikely. Indeed, in language comprehension, the brain areas that are sensitive to simple two-word composition (e.g., Pallier et al, 2011;Shain et al, 2021b) are also engaged for the processing of non-local dependencies (e.g., Blank et al, 2016;Shain, Blank et al, 2020;Shain et al, 2021a).…”

“…To illuminate the contribution of the language-selective network to language production, we use fMRI to examine the responses of the language areas-defined in individual participants by an extensively validated comprehension-based language localizer (Fedorenko et al, 2010)during production tasks. To examine both phrase-structure building and lexical access using this precision fMRI approach, we adapt a paradigm that has proven fruitful in probing combinatorial and lexico-semantic processes in comprehension (e.g., Friederici et al, 2000;Humphries et al, 2007;Fedorenko et al, 2010Fedorenko et al, , 2012aPallier et al, 2011;Shain et al, 2021). In particular, we examine neural responses during spoken (Experiments 1-2) and typed (Experiment 3) production of sentences and lists of words (as well as control, nonword sequences in Experiments 1 and 3).…”

“…To illuminate the contribution of the language-selective network to lexical access and sentence generation, we used fMRI to examine responses of the language areas-defined in individual participants by an extensively validated language 'localizer' (Fedorenko et al, 2010)-to word and sentence production. Adapting a paradigm that has proven fruitful in probing comprehension (e.g., Friederici et al, 2000;Humphries et al, 2006;Fedorenko et al, 2010Fedorenko et al, , 2012aPallier et al, 2011;Shain et al, 2021b), we examine neural responses during the production of sentences, lists of words, and nonword sequences. Brain areas that support lexical retrieval should work harder (exhibit stronger BOLD responses) when words must be accessed (word-list production) compared to a simple articulatory task (nonword production), given the additional computations required for word retrieval.…”