Perceptual audio rendering of complex virtual environments

Abstract

Figure 1: Left, an overview of a test virtual environment, containing 174 sound sources. All vehicles are moving. Mid-left, the magenta dots indicate the locations of the sound sources while the red sphere represents the listener. Notice that the train and the river are extended sources modeled by collections of point sources. Mid-right, ray-paths from the sources to the listener. Paths in red correspond to the perceptually masked sound sources. Right, the blue boxes are clusters of sound sources with the representatives of each cluster in grey. Combination of auditory culling and spatial clustering allows us to render such complex audio-visual scenes in real-time. AbstractWe propose a real-time 3D audio rendering pipeline for complex virtual scenes containing hundreds of moving sound sources. The approach, based on auditory culling and spatial level-of-detail, can handle more than ten times the number of sources commonly available on consumer 3D audio hardware, with minimal decrease in audio quality. The method performs well for both indoor and outdoor environments. It leverages the limited capabilities of audio hardware for many applications, including interactive architectural acoustics simulations and automatic 3D voice management for video games.Our approach dynamically eliminates inaudible sources and groups the remaining audible sources into a budget number of clusters. Each cluster is represented by one impostor sound source, positioned using perceptual criteria. Spatial audio processing is then performed only on the impostor sound sources rather than on every original source thus greatly reducing the computational cost.A pilot validation study shows that degradation in audio quality, as well as localization impairment, are limited and do not seem to vary significantly with the cluster budget. We conclude that our real-time perceptual audio rendering pipeline can generate spatialized audio for complex auditory environments without introducing disturbing changes in the resulting perceived soundfield.

Strategic environmental noise mapping: Methodological issues concerning the implementation of the EU Environmental Noise Directive and their policy implications

“…Such an approach would enable end users to experience strategic noise mapping results in a manner akin to that of an online gaming experience (see Drettakis et al 2007;Tsingos et al 2003). In this way, the end user is able to negotiate the urban environment and experience changes in environmental noise in a more realistic fashion.…”

Maintaining frame rate perception in interactive environments by exploiting audio-visual cross-modal interaction

“…Similarly, perceptual factors were considered in some research on audio rendering [24,36,49]. However, these methods focus on a single modality and the factors influencing that modality.…”

Mobile Group Dynamics in Large-Scale Collaborative Virtual Environments

“…All users hear each other at all times, as in a typical audio conference. Other CVEs use much more realistic audio, such as the binaural sound system used by Tsingos et al [18] in which a user wearing headphones can pinpoint the source of each sound. The advantage of RAT style audio is that users can continue communicating wherever they are in the environment.…”