Interpolating the Directional Room Impulse Response for Dynamic Spatial Audio Reproduction

Abstract

Virtual reality (VR) is increasingly important for exploring the real world, which has partially moved to virtual workplaces. In order to create immersive presence in a simulated scene for humans, VR needs to reproduce spatial audio that describes three-dimensional acoustic characteristics in the counterpart physical environment. When the user moves, this reproduction should be dynamically updated, which provides practical challenges because the bandwidth for continuously transmitting audio and video scene data may be limited. This paper proposes an interpolation approach for dynamic spatial audio reproduction using acoustic characteristics of direction and reverberation at limited numbers of positions, which are represented using a first order Ambisonics encoding of the room impulse response (RIR), called the directional RIR (DRIR). We decompose two known DRIRs into reflection components, before interpolating early dominant components for DRIR synthesis and utilizing DRIR recordings for accuracy evaluation. Results indicate that the most accurate interpolation is obtained by the proposed method over two comparative approaches, particularly in a simulated small room where most direction of arrival estimation errors of early components are below five degrees. These findings suggest precise interpolated DRIRs with limited data using the proposed approach, which is vital for dynamic spatial audio reproduction for VR applications.

Interpolation of Spatial Room Impulse Responses Using Partial Optimal Transport

“…The extended cost matrix C is generated according to (4) and ( 6), against squared Euclidean distance between real points, and with ξ = 1.05∥s P − s Q ∥ 2 2 , as the distance of the receiver movement is the maximum expected change in position for any image source. The optimization problem (7) was implemented in MATLAB 2020b [14] using the SeDuMi solver in CVX [15]. Entries in optimal T below 1 × 10 −5 are set to zero to ensure a sparse set of mappings.…”

“…One way of achieving physically more correct interpolation is to first decompose the early SRIR into localized sound events, corresponding to direct sound and early reflections, using ToAs and DoAs estimated from existing parametric SRIR processing methods [1,2,3]. Interpolation then requires finding appropriate mappings between corresponding sound events [4,5,6,7]. In [8,9], dynamic time warping is used for simplifying this task.…”

Auralization of Measured Room Transitions in Virtual Reality

“…For interpolating SRIRs, sometimes called directional RIRs (DRIRs), a first-order interpolation method is presented in [18], which separates input SRIRs into specular parts, which are the direct sound and early reflections, and the diffuse parts. These are interpolated separately, where the specular parts are interpolated individually using direction-of-arrival (DoA) estimations.…”

Best Distance Perception in Virtual Audiovisual Environment

“…In order to obtain the same spatial listening perception with the natural environment, sound-processing technology on recording, production, and replay has achieved a great progress [9], and the concept of virtual sound environment has been proposed. Virtual sound environment includes two aspects, the narrowly de ned concept and broad concept [10].…”