A shared linguistic space for transmitting our thoughts from brain to brain in natural conversations

Abstract

Effective communication hinges on a mutual understanding of word meaning in different contexts. The embedding space learned by large language models can serve as an explicit model of the shared, context-rich meaning space humans use to communicate their thoughts. We recorded brain activity using electrocorticography during spontaneous, face-to-face conversations in five pairs of epilepsy patients. We demonstrate that the linguistic embedding space can capture the linguistic content of word-by-word neural alignment between speaker and listener. Linguistic content emerged in the speaker's brain before word articulation, and the same linguistic content rapidly reemerged in the listener's brain after word articulation. These findings establish a computational framework to study how human brains transmit their thoughts to one another in real-world contexts.

It takes a village: A multi-brain approach to studying multigenerational family communication

“…These include personality traits, (social) engagement, mental states, the nature and quality of the relationship, priors, and, critically: individual neurobiological variation (“neural profiles”). For example, our work and that of others has found that inter-brain coupling between speakers and listeners is affected by sharing linguistic predictions, stimulus entrainment, and social relationships ( Bevilacqua et al, 2019 , Dikker et al, 2014 , Zada et al, 2023 , Hoehl et al, 2021 ). Research has further shown that turn-taking dynamics during verbal exchanges predict interpersonal neural coupling both in same-age and cross-age dyads ( Pan et al, 2020 , Nguyen et al, 2021 ), highlighting the importance of not only studying dynamic interactions, but also of examining coupling dynamics within such interactions.…”

Aligning Brains into a Shared Space Improves their Alignment to Large Language Models

“…Many recent studies have begun to employ encoding models to predict neural responses during natural language processing using contextual embeddings derived from LLMs (Schrimpf et al, 2021; Caucheteux & King, 2022; Goldstein et al, 2022; Toneva et al, 2022; Cai et al, 2023; Goldstein, Wang, et al, 2023; Mischler et al, 2024; Zada et al, 2023). Our study demonstrates that aligning the neural activity in each brain into a shared, stimulus-driven feature space significantly enhances encoding performance.…”

Deep social neuroscience: the promise and peril of using artificial neural networks to study the social brain

“…The use of ANNs as cognitive models is only just beginning to spread to social neuroscience ( Bolotta and Dumas, 2021 ). Neural responses to narratives and other linguistic content are being modeled using deep learning language models ( Dehghani et al ., 2017 ; Hu et al ., 2022 ; Zada et al ., 2023 ). Visual deep nets are being used as models against which the explanatory power of social features can be compared ( Dima et al ., 2022 ; McMahon et al ., 2023 ).…”