Inferring the nature of linguistic computations in the brain

Abstract

Sentences contain structure that determines their meaning beyond that of individual words. An influential study by Ding and colleagues (2016) used frequency tagging of phrases and sentences to show that the human brain is sensitive to structure by finding peaks of neural power at the rate at which structures were presented. Since then, there has been a rich debate on how to best explain this pattern of results with profound impact on the language sciences. Models that use hierarchical structure building, as well as models based on associative sequence processing, can predict the neural response, creating an inferential impasse as to which class of models explains the nature of the linguistic computations reflected in the neural readout. In the current manuscript, we discuss pitfalls and common fallacies seen in the conclusions drawn in the literature illustrated by various simulations. We conclude that inferring the neural operations of sentence processing based on these neural data, and any like it, alone, is insufficient. We discuss how to best evaluate models and how to approach the modeling of neural readouts to sentence processing in a manner that remains faithful to cognitive, neural, and linguistic principles.

The Structure and Statistics of Language Jointly Shape Cross-frequency Neural Dynamics During Spoken Language Comprehension

“…A growing number of studies demonstrated how to measure the sensitivity of the brain to naturalistic speech for word-level features. It remains difficult to control for confounding aspects, such as in Ding, Melloni, et al (2016), where it has been argued that the chunking observed around syntactic phrases might be elicited simply by word-level occurrence statistics or by the repetition of part-of-speech tags (Frank and Yang, 2018;Ten Oever, Kaushik, and Andrea E Martin, 2022).…”

“…Meyer, Sun, and Andrea E Martin, 2020b;L. Meyer, Sun, and Andrea E Martin, 2020a;Ten Oever, Kaushik, and Andrea E Martin, 2022;Haegens and Golumbic, 2018;Obleser and Kayser, 2019b). The interpretation of the role, cause and effect, of low-frequency oscillations in particular is not clear.…”

Structural and sequential regularities modulate phrase-rate neural tracking

“…Evidence shows that during successful comprehension, low-frequency neural activity becomes aligned with linguistic units such as phonemes and syllables 1,2 . This “neural tracking” effect was initially shown for these lower-level linguistic units, but more recently it has been observed for higher-level and rather abstract units such as words and phrases as well [3–9]. While syllables can be physically characterized by salient energy increases 10 , these higher-level units have no one-to-one physical correlates in the speech signal, meaning that they require endogenous computations to support neural tracking 11,12 .…”

“…Instead, the head position effect calls for a mechanism that resolves temporal inconsistency by inferring linguistic structure from the signal (cf. 9 ). A candidate mechanism of such kind is the "time-based binding" mechanism described in a computational model of neural oscillations 54 .…”

“…What is striking is that structurally organized information elicits similar effects. In particular, when phrasal units like verb phrases or noun phrases are regularly repeated in a connected speech stream, the brain response will show neural tracking at the phrase rate 3,[6][7][8][9] . Since phrase structure is organized hierarchically, these structural regularities are not directly visible in the sequential structure of language -the notion 'phrase rate' presumes the cognitive capacity to represent hierarchical phrase structure by abstracting away from the sequential input.…”