Linguistic Distributional Knowledge and Sensorimotor Grounding both Contribute to Semantic Category Production

Abstract

The human conceptual system comprises simulated information of sensorimotor experience and linguistic distributional information of how words are used in language. Moreover, the linguistic shortcut hypothesis predicts that people will use computationally cheaper linguistic distributional information where it is sufficient to inform a task response. In a pre-registered category production study, we asked participants to verbally name members of concrete and abstract categories and tested whether performance could be predicted by a novel measure of sensorimotor similarity (based on an 11-dimensional representation of sensorimotor strength) and linguistic proximity (based on word co-occurrence derived from a large corpus). As predicted, both measures predicted the order and frequency of category production but, critically, linguistic proximity had an effect above and beyond sensorimotor similarity. A follow-up study using typicality ratings as an additional predictor found that typicality was often the strongest predictor of category production variables, but it did not subsume sensorimotor and linguistic effects. Finally, we created a novel, fully grounded computational model of conceptual activation during category production, which best approximated typical human performance when conceptual activation was allowed to spread indirectly between concepts, and when candidate category members came from both sensorimotor and linguistic distributional representations. Critically, model performance was indistinguishable from typical human performance. Results support the linguistic shortcut hypothesis in semantic processing and provide strong evidence that both linguistic and grounded All three authors contributed equally to the work.

Embodied language production: sensorimotor activations and interoceptive sensibility influence which words we choose when speaking

“…However, even in cases of minimal semantic similarity the effect of sentence spatial location on the spatial properties of the produced nouns was present, indicating that the effects of sentence spatial location cannot be explained entirely by the semantic relationship between the presented sentences and the chosen sentence endings. This is in line with theoretical accounts and recent findings (Banks et al, 2021;Louwerse, 2018;Vigliocco et al, 2009).…”

The Effects of Sensorimotor and Linguistic Information on the Basic-Level Advantage

“…That is, the present findings show that the latency and accuracy of categorisation can be predicted by sensorimotor (i.e., perception-action experience of the world) and linguistic distributional information (i.e., statistical distribution of words in language) more effectively than by explaining the basic-level advantage in terms of discrete taxonomic levels. These findings are in line with recent linguistic-simulation views on the nature of concepts (e.g., Barsalou et al, 2008;Connell, 2018;Connell & Lynott, 2014;Louwerse, 2011) as well as recent categorisation research showing similar effects (e.g., Banks et al, 2021).…”

“…In sensorimotor terms, many feature-based theories emphasise that categorical distinctions emerge at least in part from commonalities in the way we perceive and interact with the world around us (Cree & McRae, 2003;Tyler et al, 2000). However, sensorimotor experience may also be considered as the extent to which a concept is experienced via each perceptual modality or action effector (i.e., sensorimotor strength: Lynott et al, 2020), where the overlap in sensorimotor experience between a category concept (e.g., dog) and a member concept (e.g., Labrador) predicts how readily people name the member as an example of that category (Banks et al, 2021). In linguistic distributional terms, the relationship between member-concept labels and category-concept labels in corpus-derived linguistic space is also an effective predictor of category membership (Banks et al, 2021;Connell & Ramscar, 2001;Riordan & Jones, 2011;Wingfield & Connell, 2022a).…”

“…However, sensorimotor experience may also be considered as the extent to which a concept is experienced via each perceptual modality or action effector (i.e., sensorimotor strength: Lynott et al, 2020), where the overlap in sensorimotor experience between a category concept (e.g., dog) and a member concept (e.g., Labrador) predicts how readily people name the member as an example of that category (Banks et al, 2021). In linguistic distributional terms, the relationship between member-concept labels and category-concept labels in corpus-derived linguistic space is also an effective predictor of category membership (Banks et al, 2021;Connell & Ramscar, 2001;Riordan & Jones, 2011;Wingfield & Connell, 2022a). When a category label (e.g., dog) appears in very similar context to a member concept label (e.g., Labrador), people tend to judge the member concept as an excellent example of its category (i.e., graded structure of concepts: Connell & Ramscar, 2001).…”

Abstract Concepts, Social Interaction, and Beliefs

“…Scholars have a growing consensus that abstract concepts evoke sensorimotor experiences, although they are more detached from them than concrete concepts. Specifically, feature production tasks showed that, with abstract concepts, participants produce sensorimotor and interoceptive properties (Harpaintner et al, 2018 ; Banks et al, 2021 ). Rating tasks showed that participants judged that sensorimotor features and effectors such as the mouth and hand are involved in abstract concepts (Ghio et al, 2013 ; Villani et al, 2019 ).…”