Neural harmonics of syntactic structure

Abstract

25 Can neural activity reveal syntactic structure building processes and their violations?26 To verify this, we recorded electroencephalographic and behavioral data as 27 participants discriminated concatenated isochronous sentence chains containing 28 only grammatical sentences (regular trials) from those containing ungrammatical 29 sentences (irregular trials). We found that the repetition of abstract syntactic 30 categories generates a harmonic structure of their period independently of stimulus 31 rate, thereby separating endogenous from exogenous neural rhythms. Behavioral 32 analyses confirmed this dissociation. Internal neural harmonics extracted from 33 regular trials predicted participants' grammatical sensitivity better than harmonics 34 extracted from irregular trials, suggesting a direct reflection of grammatical 35 sensitivity. Instead, entraining to external stimulus rate scaled with task sensitivity 36 only when extracted from irregular trials, reflecting attention-capture processing.37 Neural harmonics to repeated syntactic categories constitute the first behaviorally 38 relevant, purely internal index of syntactic competence. 39 40 42 43 Introduction 44 Speech and language are related but distinct dimensions of verbal communication 45 [1 -4]. While temporal regularities guide speech chunking in ways that are relevant 46 for language production and comprehension [3,[5][6][7][8][9][10][11][12][13]), the core of language 47 processing lies in the internal and tacit knowledge of syntactic rules, whose function 48 is to determine how words combine into meaningful utterances [14]. Isolating a direct 3 49 neural correlate of such linguistic competence has proven an extremely difficult task.50 A growing body of literature has linked brain rhythms to various aspects of language 51 comprehension. Recently, Ding and colleagues [15] tried to capture a neural 52 signature of structure building in the frequency domain by having native speakers of 53 English and Mandarin Chinese listen to continuous spoken sentences, concatenated 54 as continuous trains of monosyllabic words presented at a fixed rate of 4 Hz. They 55 tested sentences composed of two-word noun phrases (NP = a phrase headed by a 56 noun, such as "My shoe") followed by two-word verb phrases (VP = "is wet", see 57 Figure 1a). The lexical input therefore appeared at a rate of 4 Hz, phrasal units at 258 Hz, and sentences at 1 Hz. This design was selected to allow for 'frequency tagging' 59 of sentence building processes, such that the neural data could be decomposed into 60 linguistically relevant rhythmic response components. Remarkably, a Fast Fourier 61 Transform (FFT) analysis highlighted significant peaks of spectral energy emerging 62 not only at 4 Hz (word rate), but also at 2 Hz and 1 Hz. The authors deemed the 63 neural responses as a direct window into the linguistic hierarchy: 2 Hz rhythms (2-64 word units) tracked the phrases, and 1 Hz rhythms (4-word units) tracked the 65 sentences. Such linguistic rhythms emerged only if participants l...

Overt and covert prosody are reflected in neurophysiological responses previously attributed to grammatical processing

“…Keeping in mind the absence of the ½ sequence peak in our pilot nonword experiment, it is implausible that a harmonic account could explain the ½ sentence peak. Covert prosody can, moreover, arguably account for some aspects of the recent frequency tagging results investigating the relationship between harmonic structure and sentence grammaticality (Tavano et al, 2020).…”

Natural language syntax complies with the free-energy principle

“…A challenge for future research is to appropriately frame the interfacing of syntax with other systems in terms that accord with the minimisation of surprise and variational free energy. Since the FEP has attendant process theories (e.g., active inference), one of the latent pay offs of our suggestions here is the development of generative models of active inference that fully ground specific factors in syntactic theory (such as phrase boundary sensitivity, structure-dependent rules) and, through simulation work, may align with recent advances in the electrophysiology and neural dynamics and harmonics of syntax (Brennan & Martin 2020, Kaufeld et al 2020, Keitel et al 2017, Tavano et al 2021. For example, one crucial factor in any syntax model seems to be phrasal category information since it appears to drive cortical tracking of hierarchical structures (Burroughs et al 2021), in line with assumptions and predictions in work highlighting the unique computational contribution of phrase structure labeling (Adger 2013, Hornstein 2009).…”

“…One implication of this research avenue is that models of syntactic computation grounded in these dynamics (e.g., Giraud 2020, Kaufeld et al 2020, Tavano et al 2021) can be said to comply with foundational principles of the FEP (see also Guevara Erra et al 2017). For instance, endogenous low-frequency (delta) tracking of syntactic nodes (Kaufeld et al 2020) could be seen as emerging as a direct function of generative belief updating in accord with active inference, supplementing the association of delta oscillations with the cortical computations responsible for creating hierarchical linguistic structures.…”

“…Conceptually speaking, entropy reduction measures and Kolmogorov complexity measures speak to the costs of endogenous syntactic computations, being more directly informative about internal states than more externalist measures like surprisal(Murphy 2021).A challenge for future research is to appropriately frame the interfacing of syntax with other systems in terms that accord with the minimisation of surprise and variational free energy. Since the FEP has attendant process theories (e.g., active inference), one of the latent payoffs of our suggestions here is the development of generative models of active inference that fully ground specific factors in syntactic theory and, through simulation work, may align with recent advances in the electrophysiology, neural dynamics and neural harmonics of syntax(Keitel et al 2017, Tavano et al 2021. For example, one crucial factor in any syntax model seems to be phrasal category information, since it appears to drive cortical tracking of hierarchical structures(Benítez-Burraco & Murphy 2019, Burroughs et al 2021, Murphy 2020b, Murphy, Hoshi et al 2022, Wilkinson & Murphy 2016, in line with work highlighting 50 the unique computational contribution of phrase structure labeling(Adger 2013, Hornstein 2009, Murphy 2015b, Zhao 2021).…”

Inferring the nature of linguistic computations in the brain

“…Ultimately, every new discovery such as the Ding and colleagues [ 1 ] study leads to new questions. Models should be updated when new experimental data that can differentiate between proposed accounts (see, e.g., [ 13 , 47 , 48 ]) becomes available. Similarly, models must be explicit about the operations that hierarchical or sequential syntactic structure building draws upon and how they might operate in the brain in order to create predictions that can be experimentally tested [ 15 , 39 ].…”