Comprehending Sentences With the Body: Action Compatibility in British Sign Language?

Abstract

Previous studies show that reading sentences about actions leads to specific motor activity associated with actually performing those actions. We investigate how sign language input may modulate motor activation, using British Sign Language (BSL) sentences, some of which explicitly encode direction of motion, versus written English, where motion is only implied. We find no evidence of action simulation in BSL comprehension (Experiments 1-3), but we find effects of action simulation in comprehension of written English sentences by deaf native BSL signers (Experiment 4). These results provide constraints on the nature of mental simulations involved in comprehending action sentences referring to transfer events, suggesting that the richer contextual information provided by BSL sentences versus written or spoken English may reduce the need for action simulation in comprehension, at least when the event described does not map completely onto the signer's own body.

The action-sentence compatibility effect (ACE) a benchmark finding for embodiment - A meta-analysis

“…In the original study by Glenberg and Kaschak (2002), participants judged the sensibility of the sentences by pressing one of two buttons. This sensibility judgment task is in fact the most common task in this meta-analysis (Bergen & Wheeler, 2005, 2010De Scalzi et al, 2015;Gianelli, Farnè, Salemme, Jeannerod, & Roy, 2011;Glenberg, Sato, Cattaneo, Riggio, et al, 2008;Gould & Michaelis, 2018;Lugli, Baroni, Gianelli, Borghi, & Nicoletti, 2012;Papesh, 2015;Schwarzkopf, Weldle, Müller, & Konieczny, 2011;Vinson, Perniss, Fox, & Vigliocco, 2017). In a further variation participants also judged the sensibility of the sentences but this time all nonsensical sentences included a pseudoword ("You feaped the mirror"), analog to the lexical decision task where participants categorize stimuli as word or nonword (van Dam & Desai, 2017).…”

“…In some studies, researchers presented either the full sentences auditory or presented the sentence for the most part auditory, except for the transfer verb, which was displayed visually on the screen (Borreggine & Kaschak, 2006;de Vega & Urrutia, 2011;Kaschak & Borreggine, 2008). Some experiments used the exact same or a part of the original linguistic material from Glenberg and Kaschak (2002) these are Kaschak and Borreggine (2008), Papesh (2015, experiment 6,7,8) and Vinson et al (2017). Others translated the original materials to investigate the ACE in different languages (Gianelli et al, 2011) or created new linguistic materials in other languages.…”

“…Others translated the original materials to investigate the ACE in different languages (Gianelli et al, 2011) or created new linguistic materials in other languages. Included languages in the meta-analysis are English (Bergen & Wheeler, 2005, 2010Borreggine & Kaschak, 2006;De Scalzi et al, 2015;Gould & Michaelis, 2018;Kaschak & Borreggine, 2008;Papesh, 2015;van Dam & Desai, 2017;Vinson et al, 2017), German (Diefenbach et al, 2013;Schwarzkopf et al, 2011), Spanish (de Vega et al, 2013de Vega & Urrutia, 2011), Italian (Glenberg, Sato, Cattaneo, Riggio, et al, 2008;Lugli et al, 2012) and French (Gianelli et al, 2011).…”

“…The standardized signs comprise four phonological aspects -those are hand form, hand direction, place of execution, and movement. In Sign language, the visual modality is used for communication and offers a high degree of iconicity (Vinson et al, 2017). In ACE experiments using Sign language, deaf participant performed a standard ACE-experiment, but instead of written or spoken sentences, they viewed videos in which a person signed the sentences.…”

Spatial reasoning in mathematics: A cross-field perspective on deaf and general education research

“…Both fields conducted research on visualspatial processing in the brain (Hauthal et al, 2013;Verdine et al, 2017), interventions promoting spatial reasoning (Marshall et al, 2016;Thomas & Harkness, 2013), and action/experiences conceptualising mathematics (Krause, 2019;Thomson et al, 2018). Studies regarding visual-spatial processing (Marschark et al, 2015), sign language (Vinson et al, 2017), and auditory deprivation (Heimler & Pavani, 2014) were exclusively addressed for deaf educationassumedly, as the field educates people whose lived experiences enable such investigations, particularly, the second and third topic. Research on factors predicting and impacting spatial reasoning (Gilligan et al, 2018) only appeared in general education, suggesting such measures require larger, more homogeneous samples, (e.g.…”