From hands to minds: Gestures promote understanding

Kang, Seok-Min; Tversky, Barbara

doi:10.1186/s41235-016-0004-9

Cited by 47 publications

(33 citation statements)

References 75 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…What is being gestured matters as well. Research on a sequence of videotaped gestures showed that watching a tutor give an explanation of a dynamic system accompanied by gestures representing the sequence of actions led to deeper understanding of the system’s operation compared to seeing gestures representing the structure of the parts (Kang & Tversky, 2016 ).…”

Section: Being Activementioning

confidence: 99%

Embodied science and mixed reality: How gesture and motion capture affect physics education

2017

View full text Add to dashboard Cite

A mixed design was created using text and game-like multimedia to instruct in the content of physics. The study assessed which variables predicted learning gains after a 1-h lesson on the electric field. The three manipulated variables were: (1) level of embodiment; (2) level of active generativity; and (3) presence of story narrative. Two types of tests were administered: (1) a traditional text-based physics test answered with a keyboard; and (2) a more embodied, transfer test using the Wacom large tablet where learners could use gestures (long swipes) to create vectors and answers. The 166 participants were randomly assigned to four conditions: (1) symbols and text; (2) low embodied; (3) high embodied/active; or (4) high embodied/active with narrative. The last two conditions were active because the on-screen content could be manipulated with gross body gestures gathered via the Kinect sensor. Results demonstrated that the three groups that included embodiment learned significantly more than the symbols and text group on the traditional keyboard post-test. When knowledge was assessed with the Wacom tablet format that facilitated gestures, the two active gesture-based groups scored significantly higher. In addition, engagement scores were significantly higher for the two active embodied groups. The Wacom results suggest test sensitivity issues; the more embodied test revealed greater gains in learning for the more embodied conditions. We recommend that as more embodied learning comes to the fore, more sensitive tests that incorporate gesture be used to accurately assess learning. The predicted differences in engagement and learning for the condition with the graphically rich story narrative were not supported. We hypothesize that a narrative effect for motivation and learning may be difficult to uncover in a lab experiment where participants are primarily motivated by course credit. Several design principles for mediated and embodied science education are proposed.Electronic supplementary materialThe online version of this article (doi:10.1186/s41235-017-0060-9) contains supplementary material, which is available to authorized users.

show abstract

Section: Being Activementioning

confidence: 99%

Embodied science and mixed reality: How gesture and motion capture affect physics education

2017

View full text Add to dashboard Cite

show abstract

“…We already know that hand and arm gestures are crucially important to the communication of actions, with hand gestures in particular shown to be a key component of nonverbal communication (Cartmill, Beilock, & Goldin-Meadow, 2012; Kang & Tversky, 2016) and a significant characteristic of emotion recognition (Dael et al., 2012; Wallbott, 1998). Hand gesture recognition research both in humans (Goldin-Meadow, 1999; Krauss, Chen, & Chawla, 1996; Obermeier, Dolk, & Gunter, 2012) and using human–computer interfaces (Murthy & Jadon, 2009; Wachs, Kölsch, Stern, & Edan, 2011) has demonstrated the importance of decoding such cues for effective communication.…”

Section: Introductionmentioning

confidence: 99%

Removing Hand Form Information Specifically Impairs Emotion Recognition for Fearful and Angry Body Stimuli

Ross

Flack

2019

Perception

View full text Add to dashboard Cite

Emotion perception research has largely been dominated by work on facial expressions, but emotion is also strongly conveyed from the body. Research exploring emotion recognition from the body tends to refer to “the body” as a whole entity. However, the body is made up of different components (hands, arms, trunk, etc.), all of which could be differentially contributing to emotion recognition. We know that the hands can help to convey actions and, in particular, are important for social communication through gestures, but we currently do not know to what extent the hands influence emotion recognition from the body. Here, 93 adults viewed static emotional body stimuli with either the hands, arms, or both components removed and completed a forced-choice emotion recognition task. Removing the hands significantly reduced recognition accuracy for fear and anger but made no significant difference to the recognition of happiness and sadness. Removing the arms had no effect on emotion recognition accuracy compared with the full-body stimuli. These results suggest the hands may play a key role in the recognition of emotions from the body.

show abstract

“…Although, gesture-based interactions are suitable for the interaction in a VE [37], such systems are difficult to design [37]. For a novice VR user, it is often difficult to learn and remember different gestures for interactions [21,38].…”

Section: Discussionmentioning

confidence: 99%

GIST: Gesture-free Interaction by the Status of Thumb; an interaction technique for Virtual Environments

Raees¹,

Ullah²,

Rahman³

2019

AIS

View full text Add to dashboard Cite

User interface has special importance in immersive virtual environments. Interactions based on the simple and conceivable gestures of a hand may enhance immersivity of a Virtual Environment (VE). However, due to the structural issues like small size and complex shape of human hand, recognition of hand gestures are more challenging. This work introduces a novel interaction technique to perform the basic interaction tasks by the simple movement of hand instead of distinct gestures. With an ordinary camera, the fist posture of hand is segmented out from the image stream using the optimal segmentation model. Like pressing a button with a thumb, the status of thumb is traced for the activation or deactivation of the interactions. After the activation of interaction, the trajectory of hand is followed to manipulate a virtual object about an arbitrary axis. Without training and comparison of gestures, the basic interactions required in a VE are performed by the perceptive movement of a hand. By incorporating image processing in the realm of VE, the technique is implemented in a case-study project; FIRST (Feasible Interaction by Recognizing the Status of Thumb). A group of 12 users evaluated the system in a moderate lighting condition. Outcomes of the evaluation revealed that the technique is suitable for Virtual Reality (VR) applications.

show abstract

From hands to minds: Gestures promote understanding

Cited by 47 publications

References 75 publications

Embodied science and mixed reality: How gesture and motion capture affect physics education

Embodied science and mixed reality: How gesture and motion capture affect physics education

Removing Hand Form Information Specifically Impairs Emotion Recognition for Fearful and Angry Body Stimuli

GIST: Gesture-free Interaction by the Status of Thumb; an interaction technique for Virtual Environments

Contact Info

Product

Resources

About