Predicting judged similarity of natural categories from their neural representations

Abstract

We report a combined behavioral and functional magnetic resonance imaging (fMRI) study of conceptual similarity among members of a natural category (mammals). The study examined the relationship between computed pairwise similarity of neural responses to viewed mammals (e.g. bear, camel, dolphin) and subjective pairwise similarity ratings of the same set of mammals, obtained from subjects after the scanning session. In each functional region of interest (fROI), measures of neural similarity were compared to behavioral ratings. fROIs were identified as clusters of voxels that discriminated intact versus scrambled images of mammals (no information about similarity was used to define fROIs). Neural similarity was well correlated with behavioral ratings in fROIs covering the lateral occipital complex in both hemispheres (with overlap of the fusiform and inferior temporal gyri on the right side). The latter fROIs showed greater hemodynamic response to intact versus scrambled images of mammals whereas the fROIs that failed to predict similarity showed the reverse pattern. The findings provide novel evidence that information about the fine structure of natural categories is coarsely coded in regions of the ventral visual pathway. Implications for the theory of inductive inference are discussed. Keywords Concepts; Reasoning; fMRIInterest in the neural representation of concepts has grown out of the discovery of categoryspecific memory impairments (Warrington & McCarthy, 1987;Warrington & Shallice, 1984) along with the more global degradation of conceptual knowledge known as semantic dementia (Snowden, Goulding, & Neary, 1989;Warrington, 1975). Much of the ensuing research has focused on the extent to which variation across cortical areas is better characterized in category-specific or modality-specific terms. For example, Martin, Wiggs, Ungerleider, & Haxby (1996) discovered that naming both animals and tools activates ventral temporal cortex and Broca's area, while naming only animals activated the left occipital lobe, and naming only tools activated left premotor and middle temporal cortices. (For discussion, see Caramazza &Mahon, 2003, andMartin, 2007.) © 2008 Elsevier Ltd. All rights reserved. * Corresponding author. E-mail address: mattheww@princeton.edu (M. Weber). NIH Public Access Author ManuscriptNeuropsychologia. Author manuscript; available in PMC 2011 January 24.Published in final edited form as: Neuropsychologia. 2009 February ; 47(3): 859-868. doi:10.1016/j.neuropsychologia.2008.029. NIH-PA Author ManuscriptNIH-PA Author Manuscript NIH-PA Author ManuscriptIn a recent review, Tyler et al. (2003) concluded that "the most striking finding in the neuroimaging studies of category and domain specificity is that most categories activate the same neural regions with only weak and inconsistent category-specific effects." The lack of category-specificity suggests shared substrata for diverse concepts, with the identity of a given category coded by the pattern of activation across an entire region. Most curre...

Right fusiform response patterns reflect visual object identity rather than semantic similarity

“…When comparing between studies, it is important to ensure that similarity values are based on the same mathematical formulation. For instance, Weber et al (2009) report higher Pearson correlation values than we obtained, but these pertain to a correlation between fMRI and behavioral data (secondorder similarity). To our knowledge, only Mitchell et al (2008) has reported cosine similarity values in a comparable manner.…”

Pattern‐information fMRI: New questions which it opens up and challenges which face it

“…In a study of monkeys viewing visual objects, Op de Beeck et al (2001) compared behavioural and neural population code similarity measures and found them to be somewhat congruent. In human fMRI studies, a number of recent studies have directly compared and found positive correlations between behavioural judgments of similarity and multivoxel fMRI pattern similarity in ventral temporal cortex (Op de Beeck et al, 2008;Walther et al, 2009;Weber et al, 2009), including changes in pattern-similarity following learning by the subjects (Op de Beeck et al, 2006;Li et al, 2008). Learning-related changes can also affect similarity judgments indirectly, by inducing changes in neural and behavioural discrimination criteria (Li et al, 2009).…”

Transformation from image-based to perceptual representation of materials along the human ventral visual pathway

“…A number of studies have demonstrated that various basic categories of objects (e.g., Cox andNeuroImage 57 (2011) 482-494 Savoy, 2003;Haxby et al, 2001), scenes (Peelen et al, 2009;Walther et al, 2009), facial expressions (Said et al, 2010), vocal emotions (Ethofer et al, 2009), odors (Howard et al, 2009) etc., as well as simple visual features like orientation (e.g., Haynes and Rees, 2005;Kamitani and Tong, 2005) and color (Brouwer and Heeger, 2009;Parkes et al, 2009), can be decoded from the pattern of brain activity. Moreover, by analyzing the patterns of activity in various brain regions, recent studies have been able to show that higher-order visual areas represent objects categorically and hierarchically (Kriegeskorte et al, 2008) and represent objects within categories or with particular shapes in perceptually relevant ways (Haushofer et al, 2008;Op de Beeck et al, 2008Weber et al, 2009). We therefore believed that this type of analysis had the potential to read out categorical representations of materials in the brain.…”