Finding Concepts in Brain Patterns

Abstract

Semantic memory is composed of one’s accumulated world knowledge. This includes one’s stored factual information about the real-world objects and animals, which enables one to recognize and interact with the things in one’s environment. How is this semantic information organized, and where is it stored in the brain? Newly developed functional neuroimaging (fMRI) methods have provided exciting and innovative approaches to studying these questions. In particular, several recent fMRI investigations have examined the neural bases of semantic knowledge using similarity-based approaches. In similarity models, data from direct (i.e., neural) and indirect (i.e., subjective, psychological) measurements are interpreted as proximity data that provide information about the relationships among object concepts in an abstract, high-dimensional space. Concepts are encoded as points in this conceptual space, such that the semantic relatedness between two concepts is determined by their distance from one another. Using this approach, neuroimaging studies have offered compelling insights to several open-ended questions about how object concepts are represented in the brain. This chapter briefly describes how similarity spaces are computed from both behavioral data and spatially distributed fMRI activity patterns. Then, it reviews empirical reports that relate observed neural similarity spaces to various models of semantic similarity. The chapter examines how these methods have both shaped and informed our current understanding of the neural representation of conceptual information about real-world objects.

Investigating the Extent to which Distributional Semantic Models Capture a Broad Range of Semantic Relations

“…The second and third sets come from shape and manipulation similarity ratings from a common set of nouns collected by Musz, Yee, and Thompson‐Schill (2012). The second set consists of the items rated on shape similarity.…”

“…Participants were presented with word pairs such as basketball‐tiger and cherry‐banana and were asked to “Picture the objects that the words refer to and rate them according to how likely they are to be the same color” [10]. These pairs were assembled using the stimuli from several studies (Musz et al., 2012; Yee, Ahmed, & Thompson‐Schill, 2012; Yee, Huffstetler, & Thompson‐Schill, 2011). One particular strength of the items from these studies is that, due to the selection criteria, pairs that were rated as highly similar in terms of color were unlikely to be similar in terms of function, shape, or manipulation.…”

Grounded Cognition Entails Linguistic Relativity: A Neglected Implication of a Major Semantic Theory

“…If two different concepts reliably evoke two different activity patterns in that region, the degree to which they are similar can be plotted as the distance between their corresponding points. Thus, the manner in which the region implements a whole set of concepts can be portrayed as a representational geometry that indicates how the neural correlates of those concepts are related to each other at multiple scales (Bauer & Just, 2019; Musz & Thompson‐Schill, 2019).…”

“…Several different versions of it are currently being explored, but all of them share the same procedure: a computer is first trained to associate certain kinds of stimuli with certain kinds of neural activity patterns, and then it is tested to determine how well it can classify the activity patterns elicited by new stimuli. Like RSA, although this method has been used to examine the cortical underpinnings of conceptual knowledge (Bauer & Just, 2019; Musz & Thompson‐Schill, 2019), it has not yet been employed to investigate whether modal brain systems for perception and action are configured, to a nontrivial extent, in language‐specific ways, as the GCM predicts.…”