The role of sound in inducing storytelling in immersive environments

Abstract

Sound design has been a fundamental component of audiovisual storytelling. However, with technological developments things are rapidly changing. More sensory information is available and, at the same time, the user is gaining agency upon the narrative, being offered the possibility of navigating or making other decisions. These new characteristics of immersive environments bring new challenges to storytelling in interactive narratives and require new strategies and techniques for audiovisual narrative progression. Can technology offer an immersive environment where the user has the sensation of agency, of choice, where her actions are not mediated by evident controls but subliminally induced in a way that it is ensured that a narrative is being followed? Can sound be a subliminal element that induces attentional focus on the most relevant elements for the narrative, inducing storytelling and biasing search in an audiovisual immersive environment? Herein, we present a literature review that has been guided by this prospect. With these questions in view, we present our exploration process in finding possible answers and potential solution paths. We point out that consistency, in terms coherency across sensory modalities and emotional matching may be a critical aspect.

True Crime Podcasting: Journalistic Epistemology and Boundary Marking

“…Generally, podcasters focus on more intimate conversations or immersive, and narrative storytelling (Newman & Gallo, 2019). Audience-oriented research shows that podcasts are primarily consumed alone and with headphones, which favors an immersive style to draw the listener in (Salselas & Penha, 2019;Lindgren, 2021). Dowling & Miller (2019, p. 170) argue, podcasts "have developed new narrative techniques designed to leverage innovations in sound design."…”

Designing Immersive Art Installation Highlighting Culture using Spatial Augmented Reality: Case Study in Kuching, Malaysia

“…Based on figure 1 there are two important components required to create SAR-based art installation to allow spectators experience the work in an immersive space, which are sound and visual. According to Salselas, sound can capture and focus spatial or audiovisual attention as a persuasive element [8]. Oh & Kong suggested in their findings that combining animation content with emotional attachment and VR attractions will allow visitors to experience stronger emotional connection, enhanced presence, and deeper immersion [9].…”

Stereo Inverse Brightness Modulation for Guidance in Dynamic Panorama Videos in Virtual Reality

“…In the field of visual attention guidance, many different methods have been proposed in recent years for application in images as well as videos; in non‐immersive settings [DMGB04, BMSG09, VMFS11, HKS16], in VR settings using Head‐Mounted Display (HMD) [LCH*17, GSEM17, GAM18, RAK18, GTA*19], and even in an immersive room‐scale projection system [GATM18, GTA*19]. Attention guidance as a support means may be helpful in various application scenarios, such as virtual training or remote teaching [dKJ17, FMS*19, YKB19b, YKB19a], guided exploration [LH19], multi‐monitor surveillance tasks [SKB19] or immersive story telling [SP19, SRD*19, LSGB20]. For our survey on related work, we will distinguish between passive and active methods, that is, they do or do not actively incorporate real‐time gaze tracking data.…”