Semi-Supervised Sound Source Localization Based on Manifold Regularization

Laufer-Goldshtein, Bracha; Talmon, Ronen; Gannot, Sharon

doi:10.1109/taslp.2016.2555085

Cited by 67 publications

(34 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In a supervised learning approach, the mapping between the binaural cues and the source locations is learnt from annotated data using a probabilistic piecewise affine regression model. A semi-supervised approach is proposed in [116] that uses RTF values input features in order to learn the source locations based on manifold regularization.…”

Section: ) Subspace Techniquesmentioning

confidence: 99%

The LOCATA Challenge: Acoustic Source Localization and Tracking

Evers

Löllmann

Mellmann

et al. 2020

IEEE/ACM Trans. Audio Speech Lang. Process.

128

View full text Add to dashboard Cite

The ability to localize and track acoustic events is a fundamental prerequisite for equipping machines with the ability to be aware of and engage with humans in their surrounding environment. However, in realistic scenarios, audio signals are adversely affected by reverberation, noise, interference, and periods of speech inactivity. In dynamic scenarios, where the sources and microphone platforms may be moving, the signals are additionally affected by variations in the source-sensor geometries. In practice, approaches to sound source localization and tracking are often impeded by missing estimates of active sources, estimation errors, as well as false estimates. The aim of the LOCAlization and TrAcking (LOCATA) Challenge is an open-access framework for the objective evaluation and benchmarking of broad classes of algorithms for sound source localization and tracking. This article provides a review of relevant localization and tracking algorithms and, within the context of the existing literature, a detailed evaluation and dissemination of the LOCATA submissions. The evaluation highlights achievements in the field, open challenges, and identifies potential future directions.

show abstract

Section: ) Subspace Techniquesmentioning

confidence: 99%

The LOCATA Challenge: Acoustic Source Localization and Tracking

Evers

Löllmann

Mellmann

et al. 2020

IEEE/ACM Trans. Audio Speech Lang. Process.

128

View full text Add to dashboard Cite

show abstract

“…One difficulty for ML-based methods in acoustics is the limited amount of labeled data and the complex acoustic propagation in natural environments, despite large volumes of recordings [1], [2]. This limitation has motivated recent approaches for localization based on semi-supervised learning (SSL) [15], [16].…”

Section: Introductionmentioning

confidence: 99%

Semi-Supervised Source Localization in Reverberant Environments With Deep Generative Modeling

et al. 2021

Self Cite

View full text Add to dashboard Cite

We propose a semi-supervised approach to acoustic source localization in reverberant environments based on deep generative modeling. Localization in reverberant environments remains an open challenge. Even with large data volumes, the number of labels available for supervised learning in reverberant environments is usually small. We address this issue by performing semi-supervised learning (SSL) with convolutional variational autoencoders (VAEs) on reverberant speech signals recorded with microphone arrays. The VAE is trained to generate the phase of relative transfer functions (RTFs) between microphones, in parallel with a direction of arrival (DOA) classifier based on RTF-phase. These models are trained using both labeled and unlabeled RTF-phase sequences. In learning to perform these tasks, the VAE-SSL explicitly learns to separate the physical causes of the RTF-phase (i.e., source location) from distracting signal characteristics such as noise and speech activity. Relative to existing semi-supervised localization methods in acoustics, VAE-SSL is effectively an end-to-end processing approach which relies on minimal preprocessing of RTF-phase features. As far as we are aware, our paper presents the first approach to modeling the physics of acoustic propagation using deep generative modeling. The VAE-SSL approach is compared with two signal processing-based approaches, steered response power with phase transform (SRP-PHAT) and MUltiple SIgnal Classification (MUSIC), as well as fully supervised CNNs. We find that VAE-SSL can outperform the conventional approaches and the CNN in label-limited scenarios. Further, the trained VAE-SSL system can generate new RTF-phase samples, which shows the VAE-SSL approach learns the physics of the acoustic environment. The generative modeling in VAE-SSL thus provides a means of interpreting the learned representations.

show abstract

“…Another classical SSL method, steered response power with phase transform (SRP-PHAT) [9][10][11] has been shown to not be robust to non-stationary signal like speech. Recently, SSL approaches based on deep neural networks (DNNs) have been proposed [13][14][15][16][17][18][19][20]. Most of the approaches are based on supervised learning.…”

Section: Introductionmentioning

confidence: 99%

SSLIDE: Sound Source Localization for Indoors Based on Deep Learning

Ayyalasomayajula

Bianco

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

This paper presents SSLIDE, Sound Source Localization for Indoors using DEep learning, which applies deep neural networks (DNNs) with encoder-decoder structure to localize sound sources with random positions in a continuous space. The spatial features of sound signals received by each microphone are extracted and represented as likelihood surfaces for the sound source locations in each point. Our DNN consists of an encoder network followed by two decoders. The encoder obtains a compressed representation of the input likelihoods. One decoder resolves the multipath caused by reverberation, and the other decoder estimates the source location. Experiments based on both the simulated and experimental data show that our method can not only outperform multiple signal classification (MU-SIC), steered response power with phase transform (SRP-PHAT), sparse Bayesian learning (SBL), and a competing convolutional neural network (CNN) approach in the reverberant environment but also achieve a good generalization performance.

show abstract

Semi-Supervised Sound Source Localization Based on Manifold Regularization

Cited by 67 publications

References 46 publications

The LOCATA Challenge: Acoustic Source Localization and Tracking

The LOCATA Challenge: Acoustic Source Localization and Tracking

Semi-Supervised Source Localization in Reverberant Environments With Deep Generative Modeling

SSLIDE: Sound Source Localization for Indoors Based on Deep Learning

Contact Info

Product

Resources

About