Reading between the lines: Listener’s vmPFC simulates speaker cooperative choices in communication games

Abstract

Humans have a remarkable ability to understand what is and is not being said by conversational partners. It has been hypothesized that listeners decode the intended meaning of a communicative signal by assuming speakers speak cooperatively, rationally simulating the speaker’s choice process and inverting it to recover the speaker’s most probable meaning. We investigated whether and how rational simulations of speakers are represented in the listener’s brain, by combining referential communication games with functional neuroimaging. We show that listeners’ ventromedial prefrontal cortex encodes the probabilistic inference of what a cooperative speaker should say given a communicative goal and context, even when such inferences are irrelevant for reference resolution. The listener’s striatum encodes the amount of update on intended meaning, consistent with inverting a simulated mental model. These findings suggest a neural generative mechanism, subserved by the frontal-striatal circuits, that underlies our ability to understand communicative and, more generally, social actions.

Neurocomputations of strategic behavior: From iterated to novel interactions

“…(d) Listener's vmPFC encodes the intention (e.g., referring to the blue circle)-action (e.g., selecting the expression "blue") contingencies of the speaker in a manner consistent with the model prediction, even in situations where such generative signals are not necessary for recovering the intention (e.g., three cases shown in the right panel). Adapted from Mi et al (2021). vmPFC, ventromedial prefrontal cortex strategic behavior, not only because communication has long been considered a special form of a cooperative behavior between a speaker and a listener (Grice, 1975), but also because communicative decisions are often inherently novel, and achieving mutual understanding requires flexible choices and inferences against a near-infinite variety of communicative signals and contexts.…”

“…To characterize strategic generative processes in a quantitative and neurobiologically-plausible way, one potential method is to combine tools from decision neuroscience and game theory. A recent neuroimaging study offers an example of this method and initial neural evidence for the generative hypothesis in the context of interpersonal communication (Mi et al, 2021). This study investigates how the brain reads between the lines, or decodes intentions from communicative signals in context (Figure 2).…”

The social basis of referential communication: Speakers construct physical reference based on listeners’ expected visual search.

“…Beyond utility functions, our computational model also posits that speakers use a mental model of their listener. Recent work has also found evidence that people encode communicative expectations through a mental model of the speaker, which appear to be implemented in frontal–striatal circuits and in the ventromedial prefrontal cortex; Mi et al, 2021). In the context of our work, these results provide some initial evidence for the use of mental models in communication, but leave open the question of whether, at an algorithmic level, speakers also use mental models of the listener as they speak.…”

People and paradigms: mental networks and the ventromedial prefrontal cortex

“…Looking at a more general characteristic, an MN will impose its structure upon the mind when it is triggered (Mi, 2021). Similarly, religious feelings will depend upon the context and a concept of God will only affect thought and behavior when it is triggered.…”