The neuroscience of perception has recently been revolutionized with an integrative modeling approach in which computation, brain function, and behavior are linked across many datasets and many computational models. By revealing trends across models, this approach yields novel insights into cognitive and neural mechanisms in the target domain. We here present a systematic study taking this approach to higher-level cognition: human language processing, our species’ signature cognitive skill. We find that the most powerful “transformer” models predict nearly 100% of explainable variance in neural responses to sentences and generalize across different datasets and imaging modalities (functional MRI and electrocorticography). Models’ neural fits (“brain score”) and fits to behavioral responses are both strongly correlated with model accuracy on the next-word prediction task (but not other language tasks). Model architecture appears to substantially contribute to neural fit. These results provide computationally explicit evidence that predictive processing fundamentally shapes the language comprehension mechanisms in the human brain.

12brain predictivity, with each model's untrained score predictive of its trained score. These results support the hypothesis that a 13 drive to predict future inputs may shape human language processing, and perhaps the way knowledge of language is learned 14 and organized in the brain. In addition, the finding of strong correspondences between ANNs and human representations opens 15 the door to using the growing suite of tools for neural network interpretation to test hypotheses about the human mind. 16 computational neuroscience, language comprehension, fMRI, ECoG, human brain recordings, natural language processing, artificial neural networks, deep learning Specific models accurately predict human brain activity. We found ( Fig. 2a-b) that specific models predict Pereira2018 and 114Fedorenko2016 datasets with up to 100% predictivity (see Fig. S2 for generalization to another metric) relative to the noise 115 ceiling (Methods-7, Fig. S1). The Blank2014 dataset is also reliably predicted, but with lower predictivity. Models vary 116 substantially in their ability to predict neural data. Generally, embedding models such as GloVe do not perform well on any 117 dataset. In contrast, recurrent networks such as skip-thoughts, as well as transformers such as BERT, predict large portions 118 of the data. The model that predicts the human data best across datasets is GPT2-xl, which predicts Pereira2018 and 119Fedorenko2016 at close to 100% and is among the highest-performing models on Blank2014 with 32% predictivity. These 120 scores are higher in the language network than other parts of the brain (SI-4). 121Model scores are consistent across experiments/datasets. To test the generality of the model representations, we examined the 122 consistency of model scores across datasets. Indeed, if a model does well on one dataset, it tends to also do well on other 123 datasets ( Fig. 2c), ruling out the possibility that we are picking up on spurious, dataset-idiosyncratic predictivity, and 124suggesting that the models' internal representations are general enough to capture brain responses to diverse linguistic 125 materials presented visually or auditorily, and across three independent sets of participants. Specifically, model scores 126across the two experiments in Pereira2018 (overlapping sets of participants) correlate at r=.94 (Pearson here and 127 elsewhere, p<<.00001), scores from Pereira2018 and Fedorenko2016 correlate at r=.50 (p<.001), and from Pereira2018 and 128Blank2014 at r=.63 (p<.0001). 129 130Next-word-prediction task performance predicts neural scores. In vision, ANNs that perform better on the specific task of 131 visual classification also tend to better predict responses in the primate ventral stream (Schrimpf et al., 2018; Yamins et al., 132 2014). Building on work that has established a core role for predictive processing in language (Hale, 2001; Levy, 2008a; 133 Smith & Levy, 2013) and recent findings that ANNs that perform well on a next-word prediction task (a normative task 134 known as 'languag...

Two analytic traditions characterize fMRI language research. One relies on averaging activations across individuals. This approach has limitations: because of inter-individual variability in the locations of language areas, any given voxel/vertex in a common brain space is part of the language network in some individuals but in others, may belong to a distinct network. An alternative approach relies on identifying language areas in each individual using a functional ‘localizer’. Because of its greater sensitivity, functional resolution, and interpretability, functional localization is gaining popularity, but it is not always feasible, and cannot be applied retroactively to past studies. To bridge these disjoint approaches, we created a probabilistic functional atlas using fMRI data for an extensively validated language localizer in 806 individuals. This atlas enables estimating the probability that any given location in a common space belongs to the language network, and thus can help interpret group-level activation peaks and lesion locations, or select voxels/electrodes for analysis. More meaningful comparisons of findings across studies should increase robustness and replicability in language research.