How concepts are encoded in the human brain: A modality independent, category-based cortical organization of semantic knowledge

Abstract

How conceptual knowledge is represented in the human brain remains to be determined. To address the differential role of low-level sensory-based and high-level abstract features in semantic processing, we combined behavioral studies of linguistic production and brain activity measures by functional magnetic resonance imaging in sighted and congenitally blind individuals while they performed a property-generation task with concrete nouns from eight categories, presented through visual and/or auditory modalities. Patterns of neural activity within a large semantic cortical network that comprised parahippocampal, lateral occipital, temporo-parieto-occipital and inferior parietal cortices correlated with linguistic production and were independent both from the modality of stimulus presentation (either visual or auditory) and the (lack of) visual experience. In contrast, selected modality-dependent differences were observed only when the analysis was limited to the individual regions within the semantic cortical network. We conclude that conceptual knowledge in the human brain relies on a distributed, modality-independent cortical representation that integrates the partial category and modality specific information retained at a regional level.

Are Supramodality and Cross-Modal Plasticity the Yin and Yang of Brain Development? From Blindness to Rehabilitation

“…Equally, sensory-independent responses can be impaired by transcranial magnetic stimulation (TMS)-induced lesions in task-specific “visual” areas (e.g., Noppeney, 2007 ; Collignon et al, 2011; Frasnelli et al, 2011; Kupers and Ptito, 2011; Kupers et al, 2011). More recently, the employment of multivariate pattern recognition approaches offered a novel tool to demonstrate a shared coding of specific stimulus content, such as shape, motion and action, in both sighted and congenitally blind individuals across different sensory modalities (Pietrini et al, 2004; Mahon et al, 2009; Ricciardi et al, 2013; Dormal et al, 2016; Handjaras et al, 2016). Noteworthy, the homologies in the neural patterns of stimulus representation obtained with multivariate approaches are not typically limited to a mere overlap in the spatial localization of “activated” regions, but actually do involve the intrinsic content of the neural responses, suggesting that sensory-independent representations are somehow (hard)-coded at a neural level (Ricciardi et al, 2013; Handjaras et al, 2016).…”

“…Consequently, at which level (e.g., localized area or network) and how does the supramodal representation of information occur for more complex cognitive tasks? We recently demonstrated that circumscribed brain areas retain a modality-dependent processing of simple unisensory information, whereas larger networks are able to integrate the semantic content of sensory information and to generate a modality-independent representation that matches language and retains the most precise definition of concepts (Handjaras et al, 2016). This supramodal mechanism of distinct levels of stimulus processing may explain how information progresses from a sensory-based towards a more abstract conceptual representation (Mahon and Caramazza, 2011; Ricciardi and Pietrini, 2011; Ricciardi et al, 2013; Handjaras et al, 2016).…”

“…We recently demonstrated that circumscribed brain areas retain a modality-dependent processing of simple unisensory information, whereas larger networks are able to integrate the semantic content of sensory information and to generate a modality-independent representation that matches language and retains the most precise definition of concepts (Handjaras et al, 2016). This supramodal mechanism of distinct levels of stimulus processing may explain how information progresses from a sensory-based towards a more abstract conceptual representation (Mahon and Caramazza, 2011; Ricciardi and Pietrini, 2011; Ricciardi et al, 2013; Handjaras et al, 2016). From a translational perspective, we can ask how rehabilitative approaches or SSDs (which typically dissect and reproduce definite spatio-temporal features of sensory stimuli) will evolve from processing simple sensory information to processing more complex stimuli, including emotional and affective ones.…”

Illuminating the conceptual structure of the space of moral violations with searchlight representational similarity analysis

“…The present results add to a growing line of work that attempts to understand the neural organization of high-level, complex stimulus representations using more specific informational hypotheses (Chavez and Heatherton, 2015; Handjaras et al, 2016; Kriegeskorte et al, 2006). Not only can these approaches illuminate the neurocognitive correlates of these stimuli, as our results do here, but they can also provide specific organizational models to be tested and refined.…”

“…RSA and related methods have been fruitful in characterizing the structure of the space of physical object representations (Kriegeskorte et al, 2008), semantic representations (Handjaras et al, 2016; Huth et al, 2012), and lexical representations (Su et al, 2012) – as well as the key features driving structural organization. Yet their application to conceptual spaces involving social content is so far limited.…”