Shared functional specialization in transformer-based language models and the human brain

Abstract

Piecing together the meaning of a narrative requires understanding not only the individual words but also the intricate relationships between them. How does the brain construct this kind of rich, contextual meaning from natural language? Recently, a new class of artificial neural networks—based on the Transformer architecture—has revolutionized the field of language modeling. Transformers integrate information across words via multiple layers of structured circuit computations, forming increasingly contextualized representations of linguistic content. In this paper, we deconstruct these circuit computations and analyze the associated "transformations" (alongside the more commonly studied "embeddings") at each layer to provide a fine-grained window onto linguistic computations in the human brain. Using functional MRI data acquired while participants listened to naturalistic spoken stories, we find that these transformations capture a hierarchy of linguistic computations across cortex, with transformations at later layers in the model mapping onto higher-level language areas in the brain. We then decompose these transformations into individual, functionally-specialized "attention heads" and demonstrate that the emergent syntactic computations performed by individual heads correlate with predictions of brain activity in specific cortical regions. These heads fall along gradients corresponding to different layers, contextual distances, and syntactic dependencies in a low-dimensional cortical space. Our findings provide a new basis for using the internal structure of large language models to better capture the cascade of cortical computations that support natural language comprehension.

Explaining neural activity in human listeners with deep learning via natural language processing of narrative text

“…Second, we showed that it is possible to explore the internal reasoning of the "black box" of a DL language model and derive corresponding metrics to map the neural mechanism underlying language comprehension. In fact, while the saliency scores represent a useful tool for the developers to understand the behavior of a DL model, obtaining a more "explainable" artificial intelligence (AI) tool 16,27 , our findings demonstrate that these scores could be also used as neural features to shed light on neural mechanisms that are either partially unknown or simply not yet (or not completely) modeled by other metrics 15 . Furthermore, our findings are in line with a recent study demonstrating (with a different neuroimaging technique) that (i) the neural responses before the onset of a word contain valuable information about the incoming word, (ii) such information can be suitably extracted from the GPT-2 using its contextual embeddings 5 .…”

“…Recent studies have shown that DL language models based on the transformer architecture 9 , such as the GPT-2, have remarkable performances in explaining the neural correlates of language comprehension and likely share some computational principles with the human brain [4][5][6]15 . In this work, we provided further support to this hypothesis, and contributed some novel insights about two basic neural mechanisms underpinning language comprehension in humans: word prediction and context-related language processing.…”

“…First, although the GPT-2 model is one of the most biologically plausible models 7 , it has been suggested that it does not reflect the way human beings learn and manage language 43 . Thus, DL models with more human-oriented architecture and learning objective could provide a deeper understanding of language comprehension 1,2,15,43 . Second, we used a text stimulus that was a narrative and that was not familiar for all the listeners.…”

“…Thereby, future studies are warranted to assess the influence of the familiarity factor and, especially concerning the saliency scores, also their contribution to the understanding of neural computations in comparison with theoretical approaches 44 . Finally, although fMRI has been successfully used in previous studies to map language processing 4,15,30,43 , the use of high temporal resolution neuroimaging techniques, such as MEG and EEG, could…”

“…So far, previous studies have mostly focused on abstracting the neural representation of specific linguistic content from the embeddings of the GPT-2 model or on obtaining the empirical estimation of next-word probabilities from the output of the model [4][5][6] . Moreover, one preliminary study by Kumar et al 15 showed the possibility of explaining the cascade of cortical computations during language comprehension by giving in input a text to the BERT model 10 and leveraging the output of its so-called attention heads, which are the essential internal components of an attention-based DL model, that operate directly on the input words 15 . However, no previous studies considered the model "reasoning", i.e., how the model evaluates the input words and assigns the responsibility to each of them for generating a given output 16 .…”