Broadband doa estimation using convolutional neural networks trained with noise signals

Abstract

A convolution neural network (CNN) based classification method for broadband DOA estimation is proposed, where the phase component of the short-time Fourier transform coefficients of the received microphone signals are directly fed into the CNN and the features required for DOA estimation are learned during training. Since only the phase component of the input is used, the CNN can be trained with synthesized noise signals, thereby making the preparation of the training data set easier compared to using speech signals. Through experimental evaluation, the ability of the proposed noise trained CNN framework to generalize to speech sources is demonstrated. In addition, the robustness of the system to noise, small perturbations in microphone positions, as well as its ability to adapt to different acoustic conditions is investigated using experiments with simulated and real data.

Keyword Based Speaker Localization: Localizing a Target Speaker in a Multi-speaker Environment

“…We refer to this spectrum as the reverberated spectrum identifier. The reverberated speech t R c is corrupted by late reverberation which is known to have a detrimental effect on single-speaker DOA estimation [17,22]. Therefore, we also consider the spectrum |T E | of the signal containing only the direct component and the early reflections of the target signal as an identifier and call it the early spectrum identifier.…”

“…To improve the robustness of DOA estimation, deep neural networks (DNNs) have been proposed to learn a mapping between signal features and a discretized DOA space [17][18][19][20][21]. Various features such as phasemaps [17,18] and GCC-PHAT [21] have been used as inputs.…”

“…Various features such as phasemaps [17,18] and GCC-PHAT [21] have been used as inputs. In [22], the cosines and sines of the frequency-wise phase differences between microphones, termed as cosine-sine interchannel phase difference (CSIPD) features, have been shown to perform as well as phasemaps for DOA estimation, despite their lower dimensionality.…”

Quaternion Convolutional Neural Networks for Detection and Localization of 3D Sound Events

“…In this paper, we want to exploit the capabilities of both QNNs and Ambisonics to analyze 3D sounds, and in particular we focus on the localization and detection of 3D sound events. Both tasks have been widely investigated recently by using convolutional neural networks (CNNs) [19][20][21][22][23][24][25]. They are also considered as a joint task in [26] for 3D sounds, but considering each microphone signal as a separate real-valued signal.…”

Improving Reverberant Speech Separation with Binaural Cues Using Temporal Context and Convolutional Neural Networks

“…Other neural network structures, such as CNNs, are neural networks designed to process data in the form of multiple arrays (such as images with three colours channels) and contain convolutional and pooling layers [5]. CNNs have been used to estimate the DOA for speech separation in [12] and trained using synthesized noise signals, but recorded with a four-microphones array. In this paper, we present a system that is able to perform source localization and source separation.…”