Symbol Grounding Without Direct Experience: Do Words Inherit Sensorimotor Activation From Purely Linguistic Context?

Abstract

Theories of embodied cognition assume that concepts are grounded in non-linguistic, sensorimotor experience. In support of this assumption, previous studies have shown that upwards response movements are faster than downwards movements after participants have been presented with words whose referents are typically located in the upper vertical space (and vice versa for downwards responses). This is taken as evidence that processing these words reactivates sensorimotor experiential traces. This congruency effect was also found for novel words, after participants learned these words as labels for novel objects that they encountered either in their upper or lower visual field. While this indicates that direct experience with a word's referent is sufficient to evoke said congruency effects, the present study investigates whether this direct experience is also a necessary condition. To this end, we conducted five experiments in which participants learned novel words from purely linguistic input: Novel words were presented in pairs with real up- or down-words (Experiment 1); they were presented in natural sentences where they replaced these real words (Experiment 2); they were presented as new labels for these real words (Experiment 3); and they were presented as labels for novel combined concepts based on these real words (Experiment 4 and 5). In all five experiments, we did not find any congruency effects elicited by the novel words; however, participants were always able to make correct explicit judgements about the vertical dimension associated to the novel words. These results suggest that direct experience is necessary for reactivating experiential traces, but this reactivation is not a necessary condition for understanding (in the sense of storing and accessing) the corresponding aspects of word meaning.

Immediate sensorimotor grounding of novel concepts learned from language alone

“…Participants. Adapting the sample size determined by Günther et al (2018), we tested 49 native German speakers (9 identified as male, 40 as female, M Age = 23.4 years, S D Age = 3.8 years). One additional participant was not included in the analysis due to a high error rate (see the Results section).…”

“…Additionally, we employed eight different Germansounding novel words (Fente, Essede, Seige, Mende, Emahte, Greites, Gehaff, Riehrer) that were already used in the experiments by Günther et al (2018) and were not a priori associated to a vertical dimension. We constructed eight different learning items for each participant by randomly pairing these novel words with the eight novel concepts and their short descriptions (see Table 1 for examples).…”

“…In terms of the experiential trace model, zebra would be grounded indirectly, through linguistic co-occurrence with already-grounded concepts -in this case, through being described in terms of already-grounded concepts and therefore incorporating these experiential traces (compare Snefjella, Lana, & Kuperman, in press; see also Snefjella & Kuperman, 2016 for earlier versions of the same argument). Based on this assumption, Günther, Dudschig, and Kaup (2018) investigated whether novel words that were learned from language alone can lead to sensorimotor activation during language processing. In five high-powered experiments, participants performed the same test phase as in Öttl et al (2017), after learning the novel words in different, purely language-based learning paradigms that associated these words with a vertical orientation.…”

Semantic transparency is not invisibility: A computational model of perceptually-grounded conceptual combination in word processing

“…Consequently, in language processing, the linguistic stimuli would act as cues to re-activate this sensorimotor experience, or the representation formed from it. Although language typically encodes many aspects of the perceptual world, and languagebased representations can come a long way in approximating this perception-based experience (Louwerse, 2011), such redundancies are surely not perfect, and therefore not having direct access to sensorimotor experience will ultimately result in different conceptual representations and processing (Günther, Dudschig, & Kaup, 2018;Kim, Elli, & Bedny, 2019;Striem-Amit, Wang, Bi, & Caramazza, 2018).…”

Images of the unseen: Extrapolating visual representations for abstract and concrete words in a data-driven computational model

“…One prime example are abstract words such as libertarianism, jealousy, or childhood, which by definition do not refer to a distinct class of physical objects (for an overview, see Borghi et al, 2017). However, it should be noted that these issues already arise for concrete words whose referents one has never experienced directly, such as Atlantis or supernova (Günther, Dudschig, & Kaup, 2018;Günther, Nguyen, et al, 2020). The question of how can we achieve grounding in the absence of any direct sensorimotor experience is of central importance for theories of grounded cognition (Borghi et al, 2017); if they can account for only a fraction of words that are directly experienced, the usefulness and adequacy of grounded cognition theories as a generallevel cognitive theory stands in question.…”