SELD-TCN: Sound Event Localization &amp; Detection via Temporal Convolutional Networks

Guirguis, Karim; Schorn, Christoph; Guntoro, Andre; Abdulatif, Sherif; Yang, Bin

doi:10.23919/eusipco47968.2020.9287716

Cited by 53 publications

(38 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Integrating the preprocessing into the network also increases inference speed due to the efficient implementation and hardware acceleration of deep-learning frameworks. Third and last, DAS learns a task-specific representation of sound features using temporal convolutional networks (TCNs) ( Bai et al, 2018 ; van den Oord et al, 2016 ; Guirguis et al, 2021 ; Figure 1—figure supplement 1A–E ). At the core of TCNs are so-called dilated convolutions ( Yu and Koltun, 2016 ).…”

Section: Resultsmentioning

confidence: 99%

Fast and accurate annotation of acoustic signals with deep neural networks

Steinfath

Palacios-Muñoz

Rottschäfer

et al. 2021

eLife

View full text Add to dashboard Cite

Acoustic signals serve communication within and across species throughout the animal kingdom. Studying the genetics, evolution, and neurobiology of acoustic communication requires annotating acoustic signals: segmenting and identifying individual acoustic elements like syllables or sound pulses. To be useful, annotations need to be accurate, robust to noise, and fast.We here introduce DeepAudioSegmenter (DAS), a method that annotates acoustic signals across species based on a deep-learning derived hierarchical presentation of sound. We demonstrate the accuracy, robustness, and speed of DAS using acoustic signals with diverse characteristics from insects, birds, and mammals. DAS comes with a graphical user interface for annotating song, training the network, and for generating and proofreading annotations. The method can be trained to annotate signals from new species with little manual annotation and can be combined with unsupervised methods to discover novel signal types. DAS annotates song with high throughput and low latency for experimental interventions in realtime. Overall, DAS is a universal, versatile, and accessible tool for annotating acoustic communication signals.

show abstract

Section: Resultsmentioning

confidence: 99%

Fast and accurate annotation of acoustic signals with deep neural networks

Steinfath

Palacios-Muñoz

Rottschäfer

et al. 2021

eLife

View full text Add to dashboard Cite

show abstract

“…Integrating the preprocessing into the network also increases inference speed due to the efficient implementation and hardware acceleration of deep-learning frameworks. Third and last, DAS learns a task-specific representation of sound features using temporal convolutional networks (TCNs) ( Bai et al, 2018 ; van den Oord et al, 2016 ; Guirguis et al, 2021 ) ( Figure 1–Figure Supplement 1 A-E). At the core of TCNs are so-called dilated convolutions ( Yu and Koltun, 2016 ).…”

Section: Resultsmentioning

confidence: 99%

Fast and accurate annotation of acoustic signals with deep neural networks

Steinfath

Palacios

Rottschäfer

et al. 2021

Preprint

View full text Add to dashboard Cite

Acoustic signals serve communication within and across species throughout the animal kingdom. Studying the genetics, evolution, and neurobiology of acoustic communication requires annotating acoustic signals: segmenting and identifying individual acoustic elements like syllables or sound pulses. To be useful, annotations need to be accurate, robust to noise, fast. We introduce DeepSS, a method that annotates acoustic signals across species based on a deep-learning derived hierarchical presentation of sound. We demonstrate the accuracy, robustness, and speed of DeepSS using acoustic signals with diverse characteristics: courtship song from flies, ultrasonic vocalizations of mice, and syllables with complex spectrotemporal structure from birds. DeepSS comes with a graphical user interface for annotating song, training the network, and for generating and proofreading annotations (available at https://janclemenslab.org/deepss). The method can be trained to annotate signals from new species with little manual annotation and can be combined with unsupervised methods to discover novel signal types. DeepSS annotates song with high throughput and low latency, allowing realtime annotations for closed-loop experimental interventions.

show abstract

“…In this survey paper, we do not address the problem of tracking on its own, which is usually done in a separate algorithm using the sequence of DoA estimates obtained by applying SSL on successive time windows [67]. Still, several deep-learningbased SSL systems are shown to produce more accurate localization of moving sources when they are trained on a dataset that includes this type of sources [68], [69], [70], [71]. In other cases, as the number of real-world datasets with moving sources is limited and the simulation of signals with moving sources is cumbersome, a number of systems are trained on static sources, but are also shown to retain fair to good performance on moving sources [63], [72].…”

Section: Moving Sourcesmentioning

confidence: 99%

A Survey of Sound Source Localization with Deep Learning Methods

Grumiaux,

Kitić,

Girin

et al. 2021

Preprint

View full text Add to dashboard Cite

This article is a survey on deep learning methods for single and multiple sound source localization. We are particularly interested in sound source localization in indoor/domestic environment, where reverberation and diffuse noise are present. We provide an exhaustive topography of the neural-based localization literature in this context, organized according to several aspects: the neural network architecture, the type of input features, the output strategy (classification or regression), the types of data used for model training and evaluation, and the model training strategy. This way, an interested reader can easily comprehend the vast panorama of the deep learning-based sound source localization methods. Tables summarizing the literature survey are provided at the end of the paper for a quick search of methods with a given set of target characteristics.

show abstract

SELD-TCN: Sound Event Localization & Detection via Temporal Convolutional Networks

Cited by 53 publications

References 21 publications

Fast and accurate annotation of acoustic signals with deep neural networks

Fast and accurate annotation of acoustic signals with deep neural networks

Fast and accurate annotation of acoustic signals with deep neural networks

A Survey of Sound Source Localization with Deep Learning Methods

Contact Info

Product

Resources

About