Reconsidering Human Capacity for Location-Aware Audio Pattern Recognition: A Case for Immersive Exocentric Sonification

Abstract

The following paper presents a cross-disciplinary snapshot of 21st century research in sonification and leverages the review to identify a new immersive exocentric approach to studying human capacity to perceive spatial aural cues. The paper further defines immersive exocentric sonification, highlights its unique affordances, and presents an argument for its potential to fundamentally change the way we understand and study the human capacity for location-aware audio pattern recognition. Finally, the paper describes an example of an externally funded research project that aims to tackle this newfound research whitespace.

Introduction to Sonification

“…It may now seem obvious that one of the greatest advantages of sonification is its coupling with visual and other sensory inputs, with the recent revolution in virtual/augmented/mixed reality technologies being an obvious target. However, despite decades of research, sonification is still a nascent field (Bukvic and Earle 2018), and as such stands to benefit from the early-stage exploration and codifying of its unique affordances in isolation from other sensory inputs that may conflate any such assessment. As was the case with visualization, doing so may help us identify new approaches to pattern recognition and knowledge building that in turn may be applicable to other domains.…”

“…This has yielded important information about individual sensory mechanisms and has paved a way towards more holistic approaches to studying perception that have built upon this foundational knowledge. Today, we are witnessing a growing focus on the holistic study of human perception and its impact on various scenarios (Lindstrom 2005), (Bukvic and Earle 2018).…”

“…A handful of more recent research projects have tried to integrate hearing with other senses, such as head rotation, or body position, both of which are indispensable variables to accurately locating sound sources. A few studies have dabbled with using HDLAs to avoid the challenges and idiosyncrasies inherent in simulating spatial audio (Bukvic and Earle 2018).…”

“…We can refer to such an environment as the immersive exocentric environment. Consequently, we can refer to the sonification that takes place in it as immersive exocentric sonification (Bukvic and Earle 2018). Unlike the use of the term exocentric within the context of immersive environments that suggests an out-of-body experience (Dede 2009), here we refer to a sonification that emanates from a perimeter of the space and does not require the user to remain in the center of the space where the distance and the ensuing loudness between individual loudspeakers and the listener's ears may be near identical.…”

“…If we were to consider the ensuing loudspeaker front or perimeter as one canvas, we could project and move data across it. The most obvious data choices may be inherently spatial data, such as geographical or geospace data that limit the need for arbitrary assignment of variables to the spatial component (Bukvic and Earle 2018). Much like the aforesaid leveraging of the psychoacoustic meaning of environmental sounds, the immersive exocentric sonification is an entirely new opportunity in the sonification research that begs further exploration.…”