Supporting the creation of non-linear everyday AR experiences in exhibitions and museums: An authoring process based on self-contained building blocks

Abstract

The use of Augmented Reality (AR) has the potential to make everyday experiences exciting and educational. For example, AR can augment exhibits in museums with animated and interactive content. The creation of this content, however, is still facing challenges. To meet these, we employ reusable, pattern-based building blocks called AR nuggets. An AR nugget implements one application pattern in a small and self-contained piece of software to provide a complete solution for recurrent AR-based experiences. For example, in the application context of museums and exhibitions, we identify superimposition or visualization of echolocation as general patterns for AR use cases. AR nuggets encapsulate AR-specific knowledge and sophisticated functionalities to support authors and reduce the authoring task to tweaking existing templates to individual exhibits. To connect AR nuggets used in different exhibition rooms, we present novel AR nuggets that encapsulate the functionalities needed for creating a path between the exhibits. Additionally, we provide examples of AR nuggets that implement a virtual character that guides visitors to exhibits of interest. With this new set of AR nuggets, spatial connections can be authored, e.g., in the form of a guided tour with interactive narration. For this authoring task, we introduce an AR nugget manager that supports authors in creating and adapting multiple non-linear AR experiences. We illustrate our approach with the creation of an everyday AR application for a museum of natural history, share our experiences and discuss to what extent our approach can mitigate authoring challenges for everyday AR applications from a museum’s point of view. This work contributes to the field of everyday AR with 1) a pattern-based authoring concept to create complex everyday AR experiences based on self-contained building blocks, 2) a set of patterns that allows for spatial connections of these to create non-linear AR content, 3) means for supporting this authoring process in the form of an AR nugget manager, 4) lessons learned in applying our authoring concept in a real application case in a museum, 5) our observation of hurdles that still prevent more widespread use of AR in everyday applications during the realization of this application case.

Chances and Limitations of Immersive Augmented Reality for Game-based Learning in Museums

“…From the very beginning, we have geared development towards the immersive Microsoft HoloLens 2, a state-of-the-art device that enables the precise tracking required for near-fit placement of digital assets in real space, especially in relation to existing physical objects. To support the content creation, we also aimed to develop modular authoring software solutions (Rau et al 2022) and codified design knowledge related to graphic effects, narrative structures and interaction patterns (Liu et al 2021). For the development of the HoloLens 2 app on which the immersive experience runs, we used Unity with the Mixed Reality Toolkit (MRTK) plugins provided by Microsoft as a basis.…”

“…For the development of the HoloLens 2 app on which the immersive experience runs, we used Unity with the Mixed Reality Toolkit (MRTK) plugins provided by Microsoft as a basis. On this platform, several exhibition-specific components were pre-configured as complex prefabs (referred to as "nuggets"), which could be used during authoring (Rau et al 2022).…”

“…In the recently completed project presentXR (2023), after developing an authoring pipeline (Rau et al 2022), we developed a first AR prototype application for head-mounted displays (Microsoft HoloLens 2) that augmented an existing animal skeleton exhibition with digital information, and finally tested it with over 100 museum visitors. We are continuing our work in an EU-funded project LoGaCulture (2023) with the aim of applying these technology experiences and insights to locative learning games for cultural heritage in general.…”