The Cortical Representation of Language Timescales is Shared between Reading and Listening

Abstract

Natural language comprehension is thought to involve the integration of low-level sensory representations into a hierarchy of increasingly high-level representations. To investigate whether in the human brain the same pathways are used to integrate both written and spoken language, we tested whether brain representations of this hierarchy are shared between written and spoken language comprehension. Different levels of the language hierarchy were operationally defined astimescales, where each timescale refers to the spectral components of a language stimulus that vary over a certain number of words. Voxelwise encoding models were used to determine how different language timescales are represented across the cerebral cortex, for each modality separately. The encoding models were estimated with fMRI BOLD data that were recorded while participants read and listened to the same set of natural language narratives in each modality (Huth et al. 2016, Deniz et al. 2019). These models reveal that the topographic organization of language timescale representations across the cortical surface is highly similar between the two modalities. Our results suggest that brain representations of language timescales are largely independent of stimulus modality.