Investigating children's spontaneous gestures when programming using TUIs and GUIs

Abstract

Spontaneous gestures produced during mathematics learning have been widely studied, however, research on the role of gesture in computing education is limited. This paper presents an investigation into children's use of spontaneous gestures when learning programming using either a tangible user interface (TUI) or a graphical user interface (GUI). The study explored the relationship between spontaneous gestures, interface type and learning outcomes in a programming lesson for primary school students aged 6-7. In the study, 34 participants engaged in a learning activity lasting approximately 37 minutes, using a TUI or a GUI. The study used a between-subjects design, and mixed methods. Pre-test and post-test data were collected, and sessions were video recorded and subsequently coded and analysed. A video analysis scheme, adapted from mathematics education research, was used to code the spontaneous gestures produced during the learning session. We found a statistically significant difference between the mean learning gains of high-frequency gesturers and low-frequency gesturers, with the top quartile showing significantly greater learning gains. There was no significant difference in the frequency of gestures between interface types. A qualitative analysis of representational gestures showed that some children use spontaneous hand gestures to demonstrate abstract computational concepts, providing evidence for the embodiment of children's offline thinking in the computing domain.

A systematic survey on embodied cognition: 11 years of research in child–computer interaction

“…One of the studies [107] that saw a difference between how children embodied technologies looked at the difference in the embodiment patterns of the children themselves rather than the technologies. They found that children who were considered as "high gesturers" during the reflective process had a higher understanding of abstract concepts compared to "low gesturers".…”

“…or programming [107,104,102], with few studies exploring the arts, such as music, independent of STEM knowledge -Andantino [88] being an excellent example of that. We find that other studies involving design and other arts only used it as a teaching method for a more STEM-related goal rather than as the sole purpose of engagement.…”

Metaphor in Mind : Programming Teachers' Knowledge and Beliefs in Action

“…The role of gestures in education has been widely researched in areas such as mathematics (Alibali & Nathan, 2012;Núñez, 2008), science (Roth & Lawless, 2002;Solvang & Haglund, 2021;Tang et al, 2022), and programming (Almjally et al, 2020;Almjally et al, 2023;Solomon et al, 2018), as well in design and metaphor studies (Arvola & Artman, 2007;Cuccio & Fontana, 2017;Iriskhanova & Cienki, 2018;Mittelberg, 2018;Mittelberg & Waugh, 2009;Müller & Tag, 2010). Nevertheless, the application of MIG-G remains sparse.…”