Unification of sentence processing via ear and eye: An fMRI study

Abstract

We present new evidence based on fMRI for the existence and neural architecture of an abstract supramodal language system that can integrate linguistic inputs arising from different modalities such that speech and print each activate a common code. Working with sentence material, our aim was to find out where the putative supramodal system is located and how it responds to comprehension challenges. To probe these questions we examined BOLD activity in experienced readers while they performed a semantic categorization task with matched written or spoken sentences that were either well-formed or contained anomalies of syntactic form or pragmatic content. On whole-brain scans, both anomalies increased net activity over non-anomalous baseline Correspondence to: Dave Braze (braze@haskins.yale.edu). Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. NIH Public Access Author ManuscriptCortex. Author manuscript; available in PMC 2012 April 1. sentences, chiefly at left frontal and temporal regions of heteromodal cortex. The anomaly sensitive sites correspond approximately to those that previous studies (Constable et al., 2004, Michael et al., 2001) have found to be sensitive to other differences in sentence complexity (object relative minus subject relative). Regions of interest (ROIs) were defined by peak response to anomaly averaging over modality conditions. Each anomaly-sensitive ROI showed the same pattern of response across sentence types in each modality. Voxel-by-voxel exploration over the whole brain based on a cosine similarity measure of common function confirmed the specificity of supramodal zones. Keywordsreading; speech; comprehension; sentence processing; functional magnetic resonance imaging Supramodal potential, or the ability to glean similar information from spoken and written forms of a message, is an essential characteristic of the language brain, making it possible to convey linguistic messages by writing as well as speech. Of course, many perceptual abilities, such as object and person perception, also exhibit supramodal potential in the sense that people can recognize the same entity on the basis of sensory input in various modalities. Supramodality in language is special in requiring for its realization a long period of learning and instruction. Clearly, the human brain is not automatically adapted for reading and writing as soon as a person is able to speak and understand speech. Yet, only in recent years has the neural architecture of the supramodal language system and its development been an object of study in its own right. In undertaking this re...

Neural correlate of the construction of sentence meaning

“…Numerous prior studies with fMRI have identified cortical regions that respond strongly (e.g., refs. 28, 37-41) and selectively (42)(43)(44) during sentence comprehension (whether presented visually or auditorily) (28,45), but fMRI lacks the temporal resolution to reveal the responses to individual words as a sentence is presented. Hence, little is known about how neural activity in specific cortical regions unfolds over the course of a sentence as the meaning of that sentence is extracted and represented.…”

The anterior left inferior frontal gyrus contributes to semantic unification

“…Semantic unification or the process by which small pieces of word-level information are combined into larger message-level representations ) is crucial to this goal. Neuroimaging studies using Positron Emission Tomography (PET) and functional Magnetic Resonance Imaging (fMRI) have revealed a number of brain regions involved in semantic unification, such as the left inferior frontal gyrus (LIFG) (Braze et al, 2011;Diaz et al, 2011;Hagoort et al, 2009;Huang et al, 2012;Rodd et al, 2005;Tesink et al, 2009;Zhu et al, 2009), anterior temporal lobe (ATL) (Crinion et al, 2003;Crinion and Price, 2005;Warren et al, 2009), posterior superior temporal gyrus (pSTG) (Friederici et al, 2009;Friederici et al, 2003) and angular gyrus (AG) (Friederici et al, 2003(Friederici et al, , 2009. Recent reviews Zhu et al, 2011) describe the LIFG as often being activated during semantic unification in fMRI studies.…”

Functional specificity for high-level linguistic processing in the human brain

“…S1) in at least 80% of subjects individually and to be robust to changes in stimulus modality (visual/auditory) (see also ref. 32), specific stimuli, and task (passive reading vs. reading with a probe task, which is used here). Furthermore, unlike the language contrasts used in some prior studies (17,21), this contrast does not confound linguistic processing and general cognitive effort, because the memory probe task is more difficult in the control (nonwords) condition.…”