Embodied science and mixed reality: How gesture and motion capture affect physics education

Abstract

A mixed design was created using text and game-like multimedia to instruct in the content of physics. The study assessed which variables predicted learning gains after a 1-h lesson on the electric field. The three manipulated variables were: (1) level of embodiment; (2) level of active generativity; and (3) presence of story narrative. Two types of tests were administered: (1) a traditional text-based physics test answered with a keyboard; and (2) a more embodied, transfer test using the Wacom large tablet where learners could use gestures (long swipes) to create vectors and answers. The 166 participants were randomly assigned to four conditions: (1) symbols and text; (2) low embodied; (3) high embodied/active; or (4) high embodied/active with narrative. The last two conditions were active because the on-screen content could be manipulated with gross body gestures gathered via the Kinect sensor. Results demonstrated that the three groups that included embodiment learned significantly more than the symbols and text group on the traditional keyboard post-test. When knowledge was assessed with the Wacom tablet format that facilitated gestures, the two active gesture-based groups scored significantly higher. In addition, engagement scores were significantly higher for the two active embodied groups. The Wacom results suggest test sensitivity issues; the more embodied test revealed greater gains in learning for the more embodied conditions. We recommend that as more embodied learning comes to the fore, more sensitive tests that incorporate gesture be used to accurately assess learning. The predicted differences in engagement and learning for the condition with the graphically rich story narrative were not supported. We hypothesize that a narrative effect for motivation and learning may be difficult to uncover in a lab experiment where participants are primarily motivated by course credit. Several design principles for mediated and embodied science education are proposed.Electronic supplementary materialThe online version of this article (doi:10.1186/s41235-017-0060-9) contains supplementary material, which is available to authorized users.

Which One Is Better Hands on or Phet Simulation on Harmonic Oscillation

“…The advantages of PhET simulation could involve the interaction among students and teachers, build the knowledge students independently, discovery and combining of prior knowledge of students and improves interest in studying students. These results indicate conformity with previous studies showed PhET simulation can improve involvement and interaction among students, educate students to have the thinking patterns of constructivism, make learning more interesting because the students can learn while playing, and visualize the concepts of science in the form of models (Jauhari et al, 2017;Johnson-Glenberg & Megowan-Romanowicz, 2017;Setiadi & Muflika, 2015).…”

Immersive VR and Education: Embodied Design Principles That Include Gesture and Hand Controls

“…Construct c has been called sense of immersion in previous articles describing the Johnson-Glenberg embodiment taxonomy for education (Johnson-Glenberg et al, 2014a;Johnson-Glenberg and Megowan-Romanowicz, 2017). Slater's lab posits that immersion is a non-subjective property of the technological system (which includes attributes like Field of View (FOV) and fidelity to environment).…”

“…This lab has researched in mixed and augmented reality spaces for science education for over a decade; the range of topics includes geography (Birchfield and Johnson-Glenberg, 2010), nutrition science (Johnson-Glenberg and Hekler, 2013;JohnsonGlenberg et al, 2014b), simple machines (Johnson-Glenberg et al, 2015), physics (Johnson-Glenberg et al, 2014a, 2016 and forces. The full article describing the electric field study and the seven learning tasks can be found at Johnson-Glenberg and Megowan-Romanowicz (2017).…”

“…• The gesture/action should be congruent, i.e., it should be well-mapped, to the content being learned (Black et al, 2012;Johnson-Glenberg and Megowan-Romanowicz, 2017). For example, the action to start a gear train spinning should be moving something in a circle, not pushing a toggle up or down.…”

“…If feeling presence connotes that the learner's body is in the virtual world, then higher presence might also correlate with higher levels of embodiment. The original, embodied taxonomy from Table 1 (Johnson-Glenberg and Megowan-Romanowicz, 2017) consisted of four delineated degrees along the continua of the three constructs. Reprinted table is open source from Johnson-Glenberg and MegowanRomanowicz (2017).…”