Sound Field Projection System using Optical See-Through Head Mounted Display

Abstract

There are various ways to grasp the spatial and temporal structures of sound field. Sound field visualization is an effective technique to understand spatial sound information. For example, acoustical holography, optical methods, and beam-forming have been proposed and studied. In recent years, augmented reality (AR) technology has rapidly developed and is now more familiar. Many sensors, display devices, and ICT technologies have been implemented in AR equipment, which enable interaction between real and virtual worlds. In this paper, we propose an AR display system, which displays the results obtained by the beam-forming method. The system consists of 16ch microphone array, real-time sound field visualization system and optical see-through head mounted display (OST-HMD). Real-time sound field visualization system analyses sound signals recorded by 16ch microphone array by beam-forming method. Processed sound pressures data are sent to OST-HMD by using transmission control protocol (TCP), and colormap is projected on real world. Settings property of real-time sound field visualization system can be changed by using virtual user interface (UI) and TCP. In addition, multi-users can experience the system by sharing sound pressures and settings property data. Using this system, users wearing OST-HMD can observe sound field information intuitively.

Study of methods for visualizing sound pressure level distribution using mixed reality and augmented reality technologies: Visualization experiments for construction sites and building spaces

“…In this study, a system was built that uses mixed reality (MR) and augmented reality (AR) technologies of optical seethrough head-mounted displays (OST-HMDs) and tablet terminals to display the sound pressure level distribution obtained by the beamforming method in a real space [2,3]. Figure 1 shows how the developed system superimposes sound pressure level distribution on real space.…”