Verbal working memory and co-speech gesture processing

The Role of Iconic Gestures in Speech Comprehension: An Overview of Various Methodologies

“…The two studies focusing on single words elicited the largest N400 component in the centro-parietal region, whereas the third study found the largest amplitude in more anterior regions. Using a dual task paradigm, Momsen et al (2020) ’s study also showed the presence of a N400 component being at its largest over anterior channels when presenting sentences. The anterior location is compatible with previous language research eliciting a larger N400 component over anterior regions when in presence of a semantic violation in a sentence context ( Hald et al, 2006 ).…”

“…This study showed a larger N400 component over central and centroparietal regions in the absence of iconic gestures ( Wu and Coulson, 2010 ). While the centroparietal effect was found in a sentence context, it was elicited by the absence of an iconic gesture, rather than an incongruent iconic gesture ( Momsen et al, 2020 ). The centroparietal regions, therefore, appear to be involved at a local integration level while anterior regions appear to deal with a global sentence-level integration.…”

“…All behavioral studies manipulating gesture-speech congruency have highlighted faster reaction times and more correct responses when participants were in presence of congruent pairs compared to incongruent pairs ( Kelly et al, 2010a , b ; Margiotoudi et al, 2014 ; Wu and Coulson, 2014 ; Kandana Arachchige et al, 2018 ; Zhao et al, 2018 ; Momsen et al, 2020 ). These results were found when there was no specific task required ( Green et al, 2009 ; Drijvers and Özyürek, 2018 ), as well as when participants were required to perform a task where they had to pay attention to the gesture ( Kelly et al, 2010b ; Margiotoudi et al, 2014 ; Cohen-Maximov et al, 2015 ; Nagels et al, 2019 ; Bohn et al, 2020 ; Özer and Göksun, 2020b ), the speech ( Ping et al, 2014 ; Wu and Coulson, 2014 ; Drijvers and Özyürek, 2018 ) or an un-related aspect ( Kelly et al, 2010a ; Kandana Arachchige et al, 2018 ; Zhao et al, 2018 ).…”

“…Given the on-line nature of gesture-speech integration, Wu and Coulson (2014) sought to investigate the involvement of working memory in gesture-speech integration. Using a dual-task paradigm, they showed that visual-spatial, but not verbal, working memory was involved in gesture-speech integration with a higher load on visual-spatial working memory affecting performances on the gesture-speech integration task ( Wu and Coulson, 2014 ; Momsen et al, 2020 ). An iconic gesture containing semantically related information ( McNeill, 1992 ), the absence of verbal working memory involvement is curious.…”

“…One potential explanation would consist of not having considered individual differences when loading the verbal working memory span. In fact, the verbal high load condition on the secondary task was completed by having participants remembering 4 numbers ( Wu and Coulson, 2014 ; Momsen et al, 2020 ). This was the same across all participants, whilst working memory abilities vary across individuals ( Jarrold and Towse, 2006 ).…”