Supervised Contrastive Embeddings for Binaural Source Localization

Abstract

Recent data-driven approaches for binaural source localization are able to learn the non-linear functions that map measured binaural cues to source locations. This is done either by learning a parametric map directly using training data, or by learning a lowdimensional representation (embedding) of the binaural cues that is consistent with the source locations. In this paper, we adopt the second approach and propose a parametric embedding to map the binaural cues to a low-dimensional space, where localization can be done with a nearest-neighbor regression. We implement the embedding using a neural network, optimized to map points that are close in the latent space (the space of source azimuths or elevations) to nearby points in the embedding. We show that the proposed embedding generalizes well in acoustic conditions different from those encountered during training, and provides better results than unsupervised embeddings previously used for localization.

Learning Deep Direct-Path Relative Transfer Function for Binaural Sound Source Localization

“…As shown in Table I, the utilized network architectures for sound source localization include fully connected (FC) neural network [14], [17]- [19], [28]- [30], CNN [20], [21], [23], recurrent neural network (RNN) [27] and CRNN [22], [25], [26]. As for the network input, the spatial features used for training include inter-channel difference features such as IPD [14], [28]- [30], IID [14], [19], [28], eigenvectors of spatial correlation matrix [18], intensity vector [22], GCC [17], [19], [27], and the spatial spectra such as spatial pseudospectrum [23]. The magnitude spectrum [21], [27], [30] can be also fed together with the spatial features to the network, but cannot be used solely.…”

“…Li et al [12], [13] used a convolutive transfer function model and an inter-frame spectral subtraction algorithm to separately suppress reverberation and noise in order to identify the DP-RTF. Tang et al [28] designed a siamese architecture to learn a low-dimensional representation of the localization cues that is consistent with the source location. Pak et al [29] and Cheng et al [30] trained DNN models to enhance the interference-contaminated IPD on the sinusoidal tracks.…”

Enhancing direct‐path relative transfer function using deep neural network for robust sound source localization

“…With the development of deep learning techniques, lots of data-driven sound source localization works are built in a supervised manner [1]. According to the role of the deep learning model plays, these methods are classified into four categories, namely signal-to-location [2], feature-to-location [3,4], spatial spectrum-to-location [5], and feature-to-feature [6,7]-based methods. Among these methods, the feature-to-feature-based method is simple and effective for improving the performance of sound source localization in noisy and reverberant environments, as it is the data driven and the extracted features can adapt to various acoustic conditions.…”

Supervised Direct-Path Relative Transfer Function Learning for Binaural Sound Source Localization

“…Deep learning has been successfully applied to the localization task recently. Under the dual-stage localization framework, deep neural network (DNN) can be used to either extract localization features [3,4], or build the mapping from the localization features to source location [5,6]. Commonly used localization feature includes inter-channel time difference (ITD) [7], inter-channel phase difference (IPD) [8], inter-channel intensity difference (IID), relative transfer function (RTF) [9,10], etc.…”