Interactive Audio Augmented Reality in Participatory Performance

Abstract

Interactive Audio Augmented Reality (AAR) facilitates collaborative storytelling and human interaction in participatory performance. Spatial audio enhances the auditory environment and supports real-time control of media content and the experience. Nevertheless, AAR applied to interactive performance practices remains under-explored. This study examines how audio human-computer interaction can prompt and support actions, and how AAR can contribute to developing new kinds of interactions in participatory performance.This study investigates an AAR participatory performance based on the theater and performance practice by theater maker Augusto Boal. It draws from aspects of multi-player audio-only games and interactive storytelling. A user experience study of the performance shows that people are engaged with interactive content and interact and navigate within the spatial audio content using their whole body. Asymmetric audio cues, playing distinctive content for each participant, prompt verbal and non-verbal communication. The performative aspect was well-received and participants took on roles and responsibilities within their group during the experience.

Speech Puzzles (Spuzzles): Engaging the Reduced, Causal, and Semantic Listening Modes for Puzzle Design in Audio Games

“…Spoken instructions are also included in the main game to describe the scene and remind players of current tasks [26]. This support can be in the form of an audio guide that introduces players to the important elements of each stage [27]. Balan et al have reviewed various educational AGs on subjects including mathematics, programming, and biology, in which speech delivers instructions, announces the tasks at hand, and explains the educational content [28].…”

Design Dimensions of Co-Located Multi-Device Audio Experiences

“…These experiences are limited to research studies for now, utilising bespoke, custom-made devices, but represent a vision for future possibilities. Examples include Bown and Ferguson's work with DIADS [36]; SoundWear [47], a study investigating the effect of non-speech sound augmentation on children's play; and Please Confirm you are not a Robot [60], a narrative-focused AAR game.…”

“…Social games [47] Please Confirm You Are Not A Robot Multiplayer AAR game experience using Bose Frames to deliver personalised audio. Social games [60]…”

Consumer behavior in augmented shopping reality. A review, synthesis, and research agenda

“…The augmented reality (AR) technology supplements the real world with virtual elements. These supplements are often visual like in the mobile game Pokémon Go, where the digital Pokémons extend the physical environment (Hamari et al, 2019), but they could also address other senses like hearing, for example through interactive audio AR in participatory performance (Nagele et al, 2021), smelling in synesthetic visualization of odors with an odor detector (e-nose; Ward et al, 2020) or tasting by a pseudo-gustatory display (Narumi et al, 2011). Several reports have recently ranked AR as one of the top 10 technology trends (Marr, 2020;Samsung Business Insights, 2020).…”