Six Challenges for Embodiment Research

Abstract

Twenty years after Barsalou’s seminal perceptual-symbols article, embodied cognition, the notion that cognition involves simulations of sensory, motor, or affective states, has moved from an outlandish proposal to a mainstream position adopted by many researchers in the psychological and cognitive sciences (and neurosciences). Though it has generated productive work in the cognitive sciences as a whole, it has had a particularly strong impact on research into language comprehension. The view of a mental lexicon based on symbolic word representations, which are arbitrarily linked to sensory aspects of their referents, was generally accepted since the cognitive revolution in the 1950s. This has radically changed. Given the current status of embodiment as a main theory of cognition, it is somewhat surprising that a close look at the literature reveals that the debate about the nature of the processes involved in language comprehension is far from settled, and key questions remain unanswered. We present several suggestions for a productive way forward.

Semantic transparency is not invisibility: A computational model of perceptually-grounded conceptual combination in word processing

“…Rather, all three experiments employed completely language-centered tasks, two of which (naming and lexical decisions) do not even, in principle, require any kind of semantic processing. As highlighted by Ostarek and Huettig (2019), these are no trivial conditions for observing effects of sensorimotor activation: These effects are often only found when sensorimotor simulation processes are explicitly encouraged, and there are serious doubts that such processes are automatic (Lebois, Wilson-Mendenhall, & Barsalou, 2015). Nevertheless, our findings are in line with results from previous studies: Petilli et al (2019) found that the visual similarity between prime-target word pairs (also measured via the VGG-F model, Chatfield et al, 2014) predicted semantic priming effects in a lexical decision task.…”

“…We investigated these different scenarios to ensure the robustness of our model predictions over different task demands, in which semantic processing is required at various degrees. In fact, evaluating potential cross-task dissociations is currently considered one of the main challenges faced by embodied cognition research (Ostarek & Huettig, 2019), where experiments typically rely on paradigms in which sensorimotor processes are explicitly probed. In this perspective, evaluating our model estimates under various experimental conditions will help establishing the routine application of the proposed vision-based compositional process.…”

Where the action could be: Speakers look at graspable objects and meaningful scene regions when describing potential actions.

“…Overall, our findings suggest that scene meaning is co-extensive with object affordances when objects that afford interaction are present in the scene, and the two dissociate when such objects are absent. It is worth noting that the role of graspability was likely amplified both by the new stimuli and task demands (i.e., the requirement to describe actions; see Ostarek & Huettig, 2019). It remains to be seen whether graspability would explain variance in attention less well compared to meaning for equivalent stimuli using a more general task (e.g., describe the scene itself, or memorize it).…”

“…However, it is unclear whether the findings generalize to everyday cognition, or if associated affordances exert greater influence when they are task-relevant (Ostarek & Huettig, 2019).…”

Earlier and More Robust Sensorimotor Discrimination of ASL Signs in Deaf Signers During Imitation

“…The human mirror neuron system (MNS) may play an important role in imitation and perception of human actions, and thus, has been the focus of a fair amount of inquiry regarding sign language (Corina & Knapp, 2006; Emmorey, 2014; Okada et al, 2016; Mole & Turner, 2017; Ostarek & Huettig, 2019). Typically, brain regions making up the MNS are recruited both during the perception and the production of actions, causing similar neural activity during observation and execution of the same action (Rizzolatti & Fabbri-Destro, 2009).…”