Building semantic memory from embodied and distributional language experience

Abstract

Humans seamlessly make sense of a rapidly changing environment, using a seemingly limitless knowledgebase to recognize and adapt to most situations we encounter. This knowledgebase is called semantic memory. Embodied cognition theories suggest that we represent this knowledge through simulation: understanding the meaning of coffee entails re-instantiating the neural states involved in touching, smelling, seeing, and drinking coffee. Distributional semantic theories suggest that we are sensitive to statistical regularities in natural language, and that a cognitive mechanism picks up on these regularities and transforms them into usable semantic representations reflecting the contextual usage of language. These appear to present contrasting views on semantic memory, but do they? Recent years have seen a push toward combining these approaches under a common framework. These hybrid approaches augment our understanding of semantic memory in important ways, but current versions remain unsatisfactory in part because they treat sensory-perceptual and distributional-linguistic data as interacting but distinct types of data that must be combined. We synthesize several approaches which, taken together, suggest that linguistic and embodied experience should instead be considered as inseparably entangled: just as sensory and perceptual systems are reactivated to understand meaning, so are experience-based representations endemic to linguistic processing; further, sensory-perceptual experience is susceptible to the same distributional principles as language experience. This conclusion produces a characterization of semantic memory that accounts for the interdependencies between linguistic and embodied data that arise across multiple timescales, giving rise to concept representations that reflect our shared and unique experiences.

Experience-driven meaning affects lexical choices during language production

“…Instead, we provide evidence that lexical choices are influenced by experientially grounded sensory meaning of space -as conveyed by the verbal context -and that these choices are modulated by distributional properties of the linguistic context. This is in line with current hybrid theories of semantic memory, which treat sensorimotor aspects and usage-based distributional aspects of language as separate -but interactingtypes of meaning (Carota et al, 2021;Davis & Yee, 2021;Vigliocco et al, 2009).…”

“…Combining experientially grounded meaning aspects with distributional semantics So-called hybrid models integrate accounts of meaning based on experiential grounding with accounts based on distributional semantics -models which have historically been treated as separate (Davis & Yee, 2021).…”

“…Furthermore, language use as reflected in large text corpora captures many aspects of our bodily and sensory experiences as we use language to communicate about them (Durda, Buchanan, & Caron, 2009). This lead to several calls for reconciling grounded and distributional accounts of meaning (Andrews, Frank, & Vigliocco, 2014;Davis & Yee, 2021;Günther et al, 2019). These theoretical considerations are confirmed by increasing evidence that sensorimotor and distributional-linguistic meaning aspects are interacting but distinct types of knowledge.…”

Embodied language production: sensorimotor activations and interoceptive sensibility influence which words we choose when speaking

“…These robust findings can be explained in the context of hybrid theories of semantic memory which integrate experientially grounded meaning aspects and usage-based distributional aspects of language as separate but interacting types of meaning (Bi, 2021;Carota et al, 2021;Davis & Yee, 2021).…”

“…as indexed by cooccurrence metrics. This implies that we make use of the experiential and linguistic contexts in which words occur when we speak and process language (Davis & Yee, 2021).…”