Adaptation of Multiple Sound Source Localization Neural Networks with Weak Supervision and Domain-adversarial Training

He, Weipeng; Motlíček, Petr; Odobez, Jean-Marc

doi:10.1109/icassp.2019.8682655

Cited by 35 publications

(28 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Randomization (using TrainS+ instead of TrainS) did not seem to further improve the ensembling results. While most unsupervised domain adaptation methods studied in this paper outperform lower bound methods which is contrary to the results presented in [11], adversarial methods are notoriously difficult to train [29]. The high variance of Lf in our results further supports this.…”

Section: Resultssupporting

confidence: 76%

“…If not, D can learn the dissimilarities and the generator will unwantedly distort its predictions to satisfy (2). Constraining the output is common for domain adaptation in SSL with, for example, entropy methods [15,16] or weak supervision [11]. However, such methods can degrade the performance by boosting incorrect low confidence predictions [11] resulting in more false positives.…”

Section: Domain Adaptation For Multiple Sound Source 2d Localizationmentioning

confidence: 99%

“…An advantage of these data-driven methods is their ability to adapt to challenging acoustic conditions. Yet, they require a large amount of training data, which is expensive to gather and label [11]. Acoustic simulators are an appealing solution as they can abundantly generate high-quality labeled datasets.…”

Section: Introductionmentioning

confidence: 99%

“…However, such methods are not suitable to our problem because entropy minimization encourages the prediction of a single source, whereas we must cater to multiple sources. In this context, He et al [11] proposed two adaptation methods compatible with multiple sources. First, a weakly supervised method conditioned on the expected number of sources, but this number is not known in our case.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Data-Efficient Framework for Real-World Multiple Sound Source 2d Localization

Moing

Vinayavekhin

Agravante

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Deep neural networks have recently led to promising results for the task of multiple sound source localization. Yet, they require a lot of training data to cover a variety of acoustic conditions and microphone array layouts. One can leverage acoustic simulators to inexpensively generate labeled training data. However, models trained on synthetic data tend to perform poorly with real-world recordings due to the domain mismatch. Moreover, learning for different microphone array layouts makes the task more complicated due to the infinite number of possible layouts. We propose to use adversarial learning methods to close the gap between synthetic and real domains. Our novel ensemble-discrimination method significantly improves the localization performance without requiring any label from the real data. Furthermore, we propose a novel explicit transformation layer to be embedded in the localization architecture. It enables the model to be trained with data from specific microphone array layouts while generalizing well to unseen layouts during inference.

show abstract

Section: Resultssupporting

confidence: 76%

Section: Domain Adaptation For Multiple Sound Source 2d Localizationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Data-Efficient Framework for Real-World Multiple Sound Source 2d Localization

Moing

Vinayavekhin

Agravante

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…The LOCATA dataset, presented as a part of IEEE-AASP Challenge on Acoustic Source Localization and Tracking, consists of audio recordings of one or two moving and up to four static sound sources, captured with a multitude of microphone arrays, with number of microphone per array ranging from 2 (binaural system using a dummy head) to 32 (Eigenmike EM32 spherical array). The shortcoming of the LOCATA dataset is that neither the room dimensions nor the distance of the origin of the coordinate system to a corner of the room is presented, which imposes a limitation of usage of the LOCATA dataset for evaluation of learning-based SSL methods, such as presented by He et al (2018He et al ( , 2019 or Chakrabarty and Habets (2019), where the model is trained on semisynthetic data, as it impossible to accurately simulate the environment matching the real-world. Also, the moving sound sources were the human subjects, walking in front of the microphone array and talking.…”

Section: Previous Workmentioning

confidence: 99%

Dataset for Evaluation of the Performance of the Methods of Sound Source Localization Algorithms Using Tetrahedral Microphone Arrays

Sakavicius

2020

Science - Future of Lithuania

View full text Add to dashboard Cite

For the development and evaluation of a sound source localization and separation methods, a concise audio dataset with complete geometrical information about the room, the positions of the sound sources, and the array of microphones is needed. Computer simulation of such audio and geometrical data often relies on simplifications and are sufficiently accurate only for a specific set of conditions. It is generally desired to evaluate algorithms on real-world data. For a three-dimensional sound source localization or direction of arrival estimation, a non-coplanar microphone array is needed.Simplest and most general type of non-coplanar array is a tetrahedral array. There is a lack of openly accessible realworld audio datasets obtained using such arrays. We present an audio dataset for the evaluation of sound source localization algorithms, which involve tetrahedral microphone arrays. The dataset is complete with the geometrical information of the room, the positions of the sound sources and the microphone array. Array audio data was captured for two tetrahedral microphone arrays with different distances between microphones and one or two active sound sources. The dataset is suitable for speech recognition and direction-of-arrival estimation, as the signals used for sound sources were speech signals.

show abstract