Mixed-Reality Geometric Algebra Animation Methods for Gamified Intangible Heritage

Abstract

In this work we propose a novel, mobile, cross-platform open framework specifically for animated, skinned and life-size augmentations of real scenes with virtual characters. Already ten years ago we had demonstrated such technologies on the site of ancient Pompeii however, with a VR/AR closed-system running on a powerful mobile workstation hidden on a backpack. Nowadays we exhibit similar functionality running on modern smartphones running on an open-source, cross-platform gamification framework. Due to the openness of this framework we integrate a modern AR markerless camera tracking toolkit coupled with our novel algorithm for animation interpolation blending based on geometric algebra, which would not be otherwise easily achievable only in commercial, off-the-self toolkits. Thus our approach allows for open-standards research in virtual character simulation as well as novel AR augmentations for the creative and cultural heritage industries. Based on recent Mixed Reality software 3D platforms and 3D content creation pipelines, intangible digital heritage environments can benefit significantly from such versatile, efficient and robust 3D gamification methodologies.

Literary Myths in Mixed Reality

“…Mixed Reality interactive applications are unique for digital heritage (Papagiannakis et al, 2005;Papagiannakis and Magnenat-Thalmann, 2007;White et al, 2007;Sylaiou et al, 2009;Engberg, 2017), as they are able to elude one's cognitive system and produce sense of being physically present in the real, as well as the virtual world. This "sense of being there" (Papagiannakis et al, 2015) is also known as Presence (Slater, 2014). Presence is considered as a paramount factor when someone interacts with MR environments, especially considering educational or digital heritage settings (Foni et al, 2010), since it gives an indication of whereas the virtual scenario has the ability to drain the subject into it.…”

“…In this way, the level of Presence is fundamental to understanding the extension of which the subject perceives the scenario as a real world experience, even though the origin and nature of this variable are still not clear. Mixed Reality (AR and VR) Presence enabling technologies (Thomas, 2012;Kapp et al, 2013;Slater, 2014;Papagiannakis et al, 2015Papagiannakis et al, , 2018 have only recently started being investigated in Serious Games applications (Kateros et al, 2015) for digital heritage (Egges et al, 2007;Kapp et al, 2013;Abrash, 2014) in terms of entertainment (Markouzis and Fessakis, 2015), education: cognition and knowledge transfer (Capuano et al, 2016), as well as learning (Pollalis et al, 2018), gamification (Papagiannakis et al, 2015;Zikas et al, 2016), engagement (Di Pietro et al, 2017), emotions (Harley et al, 2016), and enjoyment (Sylaiou et al, 2010, Brancati et al, 2015.…”

A Comparison of Gamified, Immersive VR Curation Methods for Enhanced Presence and Human-Computer Interaction in Digital Humanities

“…To realize virtual reality games, we employed the Oculus Rift HMD DK-1 and DK-2 devices and associated SDKs via two different s/w 3D simulation frameworks, the commercial Unity3D game engine and our own, GPU-powered, teaching computer graphics open-source glGA framework [2], [3], [6]. In this work we compare the process of utilizing both frameworks with the same 3D content, based on interchangeable, standard 3D formats and content creation pipelines, both for the tangible (static 3D edifices) as well as the intangible heritage (animated 3d characters and storytelling-based site curation).…”

“…The creation tool for this environment was Google SketchUp and after its completion was subsequently exported as Collada 3D file and shown in Figure 7. Regarding the intangible digital heritage content creation that we followed, the reader can refer to [2], [3].…”

“…In Section 3 we give a brief overview of the two VR frameworks and their VR and multimedia capabilities harnessed in our two applications, namely the proprietary Unity3D [14] engine and our own, open-source glGA [2]. Section 4 describes in detail our tangible (static 3D building reconstructions) heritage content creation as our intangible heritage simulations (animated virtual characters capable of reenacting plays, customs and stories) were described in [3]. In Section 5 we present the actual gamified heritage site curation for both archaeological sites, using the previously two described frameworks based on the single3D content-creation pipeline.…”