How I See What You're Saying: The Role of Gestures in Native and Foreign Language Listening Comprehension

Abstract

In what ways do native language (NL) speakers and foreign language (FL) learners differ in understanding the same messages delivered with or without gestures? To answer this question, seventh‐ and eighth‐grade NL and FL learners of English in the United States and Norway were shown a video of a speaker describing, in English, a cartoon image that the viewers could not see. For half the viewers, the speaker's gestures were visible; for the others they were not. Participants drew a picture of each description, which was later coded for recall of explicit information, comprehension of logically implied information, and distortions. Overall, NL listeners produced the most accurate drawings; the presence of gestures did not appreciably facilitate NL comprehension. In contrast, the availability of gestures had a measurable effect on FL listener performance: It enabled them to produce drawings within native‐like range. However, lack of gestures negatively impacted FL listener comprehension and recall. Regardless of condition, distortions in FL responses were significantly more frequent than in NL responses. All participants reported valuing visual cues in communication, yet their understanding of their value for their actual drawing performance was inconsistent, suggesting a difference between visual cue preference and visual cue dependence.

Beat Gestures for Comprehension and Recall: Differential Effects of Language Learners and Native Listeners

“…Then, participants were informed that they were going to perform a narrative comprehension task, and were directed to read the instructions carefully. Instructions (adapted from Dahl and Ludvigsen, 2014) were available in both French and English and were as follows:…”

“…Speech is a multimodal act that allows for listeners to make use of both auditory as well as visual cues to make sense of the incoming message. Numerous studies have shown that speech produced with referential gestures 1 boost both comprehension and recall in the L1 (Riseborough, 1981;Cohen and Otterbein, 1992; among many others), with very few studies showing no effects (e.g., Austin and Sweller, 2014;Dahl and Ludvigsen, 2014). Similarly, positive results have also been found in the L2 (Sueyoshi and Hardison, 2005;Tellier, 2008;Kelly et al, 2009;Macedonia et al, 2011; among many others).…”

“…To our knowledge, only one study has assessed the benefits of the presence of iconic gestures on recall and comprehension by native and non-native listeners using larger discourses. Dahl and Ludvigsen (2014) directly compared the effects of iconic gestures in both native and non-native listeners in terms of recall and comprehension in a cartoon picture drawing task. 28 native English speaking adolescents and 46 Norwegian adolescents who had been learning English for 7-8 years participated in the study.…”

Degree of Language Experience Modulates Visual Attention to Visible Speech and Iconic Gestures During Clear and Degraded Speech Comprehension

“…Listeners often perceive visual input that not only consists of visible speech, but also iconic hand gestures, which can convey semantic information about the speech signal. Previous work has demonstrated that listeners integrate this semantic information with speech (Kelly, Creigh, & Bartolotti, ) and that both native and non‐native listeners can benefit from semantic information that is conveyed by gestures (Dahl & Ludvigsen, ; Sueyoshi & Hardison, ), especially when speech is degraded (Drijvers & Özyürek, , , ; Drijvers et al, , ).…”

Assessing Second Language Listening Over the Past Twenty Years: A Review Within the Socio-Cognitive Framework

“…Research on the role of videos in L2 listening tests produced mixed results. Non-verbal information in videos was found to improve test scores (Ginther, 2002;Jones and Plass, 2002;Sueyoshi and Hardison, 2005;Wagner, 2010bWagner, , 2013aDahl and Ludvigsen, 2014). However, the score difference was not pronounced (Coniam, 2001;Cubilo and Winke, 2013;Batty, 2015;Suvorov, 2015).…”