Intelligent Acoustic Interfaces With Multisensor Acquisition for Immersive Reproduction

Abstract

Immersive speech communication systems have been gaining increasing attention due to their ability to reproduce enhanced acoustic images, and thus achieving good performance in terms of sound quality and accuracy. In this context, a fundamental role is played by intelligent acoustic interfaces (IAIs), which aim at acquiring and/or reproducing desired acoustic information with enhanced perception. The recent widespread availability of multimedia devices, equipped with different kind of sensors, has broadened the range of data processing methods, thus giving a chance for developing advanced IAIs. In this paper, we propose an immersive communication system composed of two IAIs: the first one exploits microphones and cameras, together with a signal processing system, to reduce unwanted noise and enhance the speech quality of the desired information in the transmitting room; the second one is an advanced reproduction system based on a loudspeaker array and on an effective wave field synthesis technique capable of reproducing the spatial perception of the desired speech source in the receiving room. The whole system has been assessed in simulated and real immersive communication scenarios: objective and subjective evaluations have been shown the effectiveness of the proposed system. Index Terms-Immersive communication, intelligent acoustic interface, kinect sensor, multichannel reproduction, multimodal interaction, noise reduction.

Multisensory Wearable Interface for Immersion and Telepresence in Robotics

“…These multimodal systems can create the sensation of full immersion that provide users with the three major sensory inputs of visual, auditory and haptic information [18], [19]. Specifically, teleoperation, telemanipulation and telepresence have been benefited from these wearable devices that, coupled to robots composed of wheels, a stand and a camera, provide humans with an enhanced control of a robot located in a remote environment [20], [21].…”

Quaternion Convolutional Neural Networks for Detection and Localization of 3D Sound Events

“…Immersive audio has changed the way people make use of audio services, placing a greater attention to the satisfaction of the user-required quality [1,2]. In this context, the last few years have been characterized by a wide spread of commercial intelligent acoustic interfaces, basically composed of acoustic interfaces equipped with intelligent signal processors [1,3,4]. This kind of devices can be found in many applications, as well as in human everyday life, such as home automation, voice assistance, safety and security by robots, audio surveillance, virtual reality in gaming and entertainment, up to speech recognition applications.…”

Convex combination of MIMO filters for multichannel acoustic echo cancellation

“…The growing availability of resources for data transmission and processing is encouraging the spread of hands-free speech communication systems which offer users an immersive experience [1]- [3]. An immersive communication scenario is characterized by the use of multiple microphones and loudspeakers, which aim at capturing and reproducing desired signals while focusing on preserving the audio quality perceived by users.…”

“…In such context, the presence of interfering sources and reverberation may cause a quality degradation of speech communications. In order to address this problems, microphones and loudspeakers are usually connected with signal processing systems, thus resulting in intelligent acoustic interfaces [3], [4]. One of the main problems of immersive scenarios is the multichannel acoustic echo cancellation (MAEC), which is caused by the multiple coupling between microphones and loudspeakers.…”