Making Tangible the Intangible: Hybridization of the Real and the Virtual to Enhance Learning of Abstract Phenomena

Abstract

Interactive systems based on augmented reality (AR) and tangible user interfaces (TUI) hold great promise for enhancing how to learn and understand abstract phenomena. In particular, they combine the advantages of numerical simulation and pedagogical support while keeping the learner involved in true physical experimentation. In this paper, we discuss three examples of such AR and TUI systems that address hardly perceivable concepts. The first example, Helios, targets K-12 learners in the field of astronomy. The second one, Hobit, is dedicated to experiments in wave optics. Finally, the third one, Teegi, allows one to get to know more about brain activity. These three hybrid interfaces have emerged from a common basis that jointly combines research and development work in the fields of instructional design and human-computer interaction (HCI), from theoretical to practical aspects. Based on investigations carried out in the context of real use and grounded on works in education and HCI, we formalize how and why the hybridization of the real and the virtual leverages the way learners understand intangible phenomena in Science education.

Could a tangible interface help a child to weigh his/her opinion on usability?

“…That is to say, the rating qualities of a system must be rendered tangible to the respondents. As demonstrated/showed/illustrated in [13], a considerable number of reviews highlight the promise of TUIs. TUI has many advantages to improve interaction accessibility, building on everyday skills and experiences [60].…”

CARDS: A Mixed-Reality System for Collaborative Learning at School

“…They suggest that physical interactions (Naive Physics) have the potential to reduce the level of abstraction (extrinsic cognitive load) and facilitate learning through a more "intuitive" (meaning : facilitating through learning acquired since the user's early age) way to interact with the interface and its content [20]. This type of interface would also promote a more active engagement, through manual activities [20,39], allowing the learning of more complex, abstract or very intangible notions [12]. In line with [32], transferring interactions with the digital world into the real one could scaffold cognition, i.e.…”

“…A first example of this association with didactic capabilities was presented by Underkoffler et al [35] where they used a video-projector to display on a tabletop a numerical simulation representing air flows that could be controlled by moving buildings miniatures. Later it was shown that this approach based on augmented physical manipulation can promote the understanding of complex and often abstract phenomena [12]. Still with the aim of facilitating the learning of complex concepts in a school context, Zufferey developed an augmented tabletop mixing SAR and TUI, called Tinker Lamp [40].…”

“…For such kinds of activities, ubiquitous approaches that take place in a three-dimensional space and foster spatial interactions and expressive representations seem to have a good potential [15]. In particular, digital approaches that are based on tangible interaction and augmented reality open new opportunities compared to more traditional desktop and tablet approaches [12]. Despite their potential, concrete implementations of such approaches at school remain rare.…”

Le numérique pour l’apprentissage collaboratif : nouvelles interfaces, nouvelles interactions

“…Elles peuvent également s'appuyer sur des interfaces numériques de nouvelle génération modifiant les modalités d'interactions humain-machine, mais aussi entre leurs utilisateurs et avec les représentations des savoirs. C'est notamment le cas pour celles qui exploitent la réalité augmentée, les objets connectés, les robots et/ou les dispositifs tactiles ou tangibles (Cuendet et al, 2013;Fleck et Hachet, 2016, Schneider et al, 2011Kubicki et al, 2014). Ces interfaces de nouvelle génération en milieu scolaire, alternatives des écrans-clavier-souris, remettent en question les attentes, les besoins et les réalités de pratiques, non seulement des élèves mais aussi de leurs enseignants, et plus largement de toute la communauté éducative.…”