Probing Word Syntactic Representations in the Brain by a Feature Elimination Method

Abstract

Neuroimaging studies have identified multiple brain regions that are associated with semantic and syntactic processing when comprehending language. However, existing methods cannot explore the neural correlates of fine-grained word syntactic features, such as part-of-speech and dependency relations. This paper proposes an alternative framework to study how different word syntactic features are represented in the brain. To separate each syntactic feature, we propose a feature elimination method, called Mean Vector Null space Projection (MVNP). This method can remove a specific feature from word representations, resulting in one-feature-removed representations. Then we respectively associate one-feature-removed and the original word vectors with brain imaging data to explore how the brain represents the removed feature. This paper for the first time studies the cortical representations of multiple fine-grained syntactic features simultaneously and suggests some possible contributions of several brain regions to the complex division of syntactic processing. These findings indicate that the brain foundations of syntactic information processing might be broader than those suggested by classical studies.

Computational Language Modeling and the Promise of In Silico Experimentation

“…In particular, the in silico paradigm can both draw upon large-scale multidisciplinary efforts to build tools and methods for interpreting neural network language models ( Ettinger et al, 2018 ; Hewitt & Manning, 2019 ; Ravfogel et al, 2020 ; Sundararajan et al, 2017 ), as well as contribute to them by providing a human neural benchmark. Furthermore, more interpretable models allow for novel causal intervention experiments that perturb and control ANNs in ways that biological neural networks cannot be perturbed ( Zhang et al, 2022 ).…”

Language Cognition and Language Computation -- Human and Machine Language Understanding

“…The author believes that language-cognition experiments combined with computational models can eliminate the research limitations above. For example, using computational models can separate different experimental variables and study the role of different language variables and cognitive functions based on neural activity data collected from natural texts [136,137,138]. With the continuous improvement of the performance of languagecomputation methods based on neural network methods, it is increasingly accurate to use models to separate different language features so that the visual and auditory perception, multimodal information fusion, and language in different regions of the brain can be calculated on the same batch of data.…”

MEG Encoding using Word Context Semantics in Listening Stories

“…(1) Basic Syntactic Features: Similar to [29,4,5], we use various multi-dimensional syntactic features such as Complexity Metrics (Node Count (NC), Word Length (WL), Word Frequency (WF)), Part-of-speech (POS) and Dependency tags (DEP), described briefly below. Node Count (NC) The node count for each word is the number of subtrees that are completed by incorporating each word into its sentence.…”

“…The central aim of brain encoding for language processing analysis is to unravel how the brain represents linguistic knowledge (i.e. semantic and syntactic properties) and carries out sentence-processing information [1,2,3,4,5] by modeling the effect of such information on brain recordings. For instance, using functional Magnetic Resonance Imaging (fMRI) brain recordings, a number of previous studies have investigated the alignment between text stimuli representations extracted from language models (e.g.…”

“…Studies using word embedding representations and fMRI have revealed that syntactic features are distributively represented across brain language networks and largely overlapped with semantic networks [4,5]. Despite the great strides in learning sentence comprehension at a functional level, there are still many problems that could benefit from further improvements in understanding sentence structure and meaning at the temporal level.…”