Platform for dynamic virtual auditory environment real-time rendering system

Abstract

This paper reports the recent works and progress on a PC and C++ language-based virtual auditory environment (VAE) system platform. By tracing the temporary location and orientation of listener's head and dynamically simulating the acoustic propagation from sound source to two ears, the system is capable of recreating free-field virtual sources at various directions and distances as well as auditory perception in reflective environment via headphone presentation. Schemes for improving VAE performance, including PCA-based (principal components analysis) near-field virtual source synthesis, simulating six degrees of freedom of head movement, are proposed. Especially, the PCA-based scheme greatly reduces the computational cost of multiple virtual sources synthesis. Test demonstrates that the system exhibits improved performances as compared with some existing systems. It is able to simultaneously render up to 280 virtual sources using conventional scheme, and 4500 virtual sources using the PCA-based scheme. A set of psychoacoustic experiments also validate the performance of the system, and at the same time, provide some preliminary results on the research of binaural hearing. The functions of the VAE system is being extended and the system serves as a flexible and powerful platform for future binaural hearing researches and virtual reality applications. Virtual auditory or acoustic environment (VAE) recreates auditory perceptions or events as those would happen in real world by controlling acoustic environment artificially. As a high-technology, VAE interdisciplines the fields of acoustics/physics, signal processing, computer science and human perception. It not only serves as an important experimental tool for the human hearing research, but also constitutes a major part of the multimedia and virtual reality technique. VAE has a lot of potential applications in various fields such as communication, room acoustic design, as well as military and aeronaut training [1]. In real environments, a complex sound field consists of direct sound waves from sound sources and reflected/ scattered sound waves from boundaries. The temporal and spatial characteristics of the sound field capture the information of sources and surrounding environment. The presence of a listener will further disturb the sound field, so that the pressures received by two ears are modified by the scattering, diffraction and reflection of the human anatomical structures. It is just these source direction-dependent scattering, diffraction and reflection that encode the temporal and spatial information of sound field into binaural signals at eardrums. In addition, source or listener's head movement alters the transmission from source to two ears and thereby alters the binaural pressures, which brings dynamic auditory information. Auditory (including high-level nerve) system analyzes and processes the binaural signals and related dynamic information, resulting in spatial auditory perception or events, such as source localization and subjective spatial per...

Typical data and cluster analysis on head-related transfer functions from Chinese subjects

“…In addition, the psychoacoustic experiments in the present work are limited in static virtual auditory display, in which the dynamic localization cues caused by head motion are omitted. A further work is to incorporate the results of the present work into dynamic virtual auditory environment [39].…”

A Comparison on the Localization Performance of Static and Dynamic Binaural Ambisonics Reproduction with Different Order

“…The experiment was conducted via a virtual auditory display (VAD) [13]. The VAD was based on a PC with windows platform and software written in C++ language.…”

Spatial Sound‐History, Principle, Progress and Challenge

“…Dynamic Virtual auditory environment (VAE) is a promising direction. Some groups have developed dynamic VAEs for scientific purposes [84][85][86][87] . With the introduction of low-cost head trackers products in recent years, dynamic VAEs have been increasingly using in the field of consumer electronics.…”