A Sequence Matching Network for Polyphonic Sound Event Localization and Detection

Nguyen, Thi Ngoc Tho; Jones, Douglas L.; Gan, Woon‐Seng

doi:10.1109/icassp40776.2020.9053045

Cited by 35 publications

(46 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 2 lists the performance of the three DcaseNet architectures when trained on a single task. The top two rows describe the results from the official DCASE baselines and recent methods [11,31], both submitted for the DCASE 2020 challenge. Note that the reference performances for TAG cannot be reported because neither an official development nor an evaluation set exists.…”

Section: Results Analysismentioning

confidence: 99%

“…Performance of proposed architectures for the performance for each task along with results from the DCASE official baseline and recent methods (higher values indicate better performance for all metrics except for ER). The performance of the reference methods are taken from [11] for ASC and [31] for SED. The values in boldface indicate the highest performance per metric.…”

Section: Experimental Configurationsmentioning

confidence: 99%

See 1 more Smart Citation

DCASENET: An Integrated Pretrained Deep Neural Network for Detecting and Classifying Acoustic Scenes and Events

Jung

Shim

Kim

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Although acoustic scenes and events include many related tasks, their combined detection and classification have been scarcely investigated. We propose three architectures of deep neural networks that are integrated to simultaneously perform acoustic scene classification, audio tagging, and sound event detection. The first two architectures are inspired by human cognitive processes. The first architecture resembles the short-term perception for scene classification of adults, who can detect various sound events that are then used to identify the acoustic scene. The second architecture resembles the long-term learning of babies, being also the concept underlying self-supervised learning. Babies first observe the effects of abstract notions such as gravity and then learn specific tasks using such perceptions. The third architecture adds a few layers to the second one that solely perform a single task before its corresponding output layer. The aim is to build an integrated system that can serve as a pretrained model to perform the three abovementioned tasks. Experiments on three datasets demonstrate that the proposed architecture, called DcaseNet, can be either directly used for any of the tasks while providing suitable results or fine-tuned to improve the performance of one task. The code and pretrained DcaseNet weights are available at https://github.com/Jungjee/DcaseNet.

show abstract

Section: Results Analysismentioning

confidence: 99%

Section: Experimental Configurationsmentioning

confidence: 99%

DCASENET: An Integrated Pretrained Deep Neural Network for Detecting and Classifying Acoustic Scenes and Events

Jung

Shim

Kim

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…For detection and classification of acoustics of scenes and events (DCASE) [22], deep learning-based methods for SELD have been proposed [18][19][20][21]. These methods simultaneously perform SSL and SED of environmental sounds containing many classes.…”

Section: Sound Event Localization and Detection Methods For Environmementioning

confidence: 99%

“…Thus, spatial features may not be used effectively. Deep learning-based methods for sound event localization and detection (SELD) have been proposed [18][19][20][21]. These methods simultaneously perform SSL and sound event detection (SED) of environmental sounds.…”

Section: Introductionmentioning

confidence: 99%

Multichannel environmental sound segmentation

et al. 2021

View full text Add to dashboard Cite

This paper proposes a multichannel environmental sound segmentation method. Environmental sound segmentation is an integrated method to achieve sound source localization, sound source separation and classification, simultaneously. When multiple microphones are available, spatial features can be used to improve the localization and separation accuracy of sounds from different directions; however, conventional methods have three drawbacks: (a) Sound source localization and sound source separation methods using spatial features and classification using spectral features trained in the same neural network, may overfit to the relationship between the direction of arrival and the class of a sound, thereby reducing their reliability to deal with novel events. (b) Although permutation invariant training used in autonomous speech recognition could be extended, it is impractical for environmental sounds that include an unlimited number of sound sources. (c) Various features, such as complex values of short time Fourier transform and interchannel phase differences have been used as spatial features, but no study has compared them. This paper proposes a multichannel environmental sound segmentation method comprising two discrete blocks, a sound source localization and separation block and a sound source separation and classification block. By separating the blocks, overfitting to the relationship between the direction of arrival and the class is avoided. Simulation experiments using created datasets including 75-class environmental sounds showed the root mean squared error of the proposed method was lower than that of conventional methods.

show abstract

“…A new dataset of spatialized sound events was generated for the task [17] and a SELDnet implementation was provided by the authors as a baseline for the challenge participants. 2 Beyond the works associated with the challenge [18]- [39], multiple works have followed aiming to address the SELD task in a new way or improve on the limitations of the challenge submissions [40]- [43].…”

Section: Introductionmentioning

confidence: 99%

Overview and Evaluation of Sound Event Localization and Detection in DCASE 2019

Politis

Mesaros

Adavanne

et al. 2021

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Sound event localization and detection is a novel area of research that emerged from the combined interest of analyzing the acoustic scene in terms of the spatial and temporal activity of sounds of interest. This paper presents an overview of the first international evaluation on sound event localization and detection, organized as a task of the DCASE 2019 Challenge. A large-scale realistic dataset of spatialized sound events was generated for the challenge, to be used for training of learning-based approaches, and for evaluation of the submissions in an unlabeled subset. The overview presents in detail how the systems were evaluated and ranked and the characteristics of the best-performing systems. Common strategies in terms of input features, model architectures, training approaches, exploitation of prior knowledge, and data augmentation are discussed. Since ranking in the challenge was based on individually evaluating localization and event classification performance, part of the overview focuses on presenting metrics for the joint measurement of the two, together with a reevaluation of submissions using these new metrics. The new analysis reveals submissions that performed better on the joint task of detecting the correct type of event close to its original location than some of the submissions that were ranked higher in the challenge. Consequently, ranking of submissions which performed strongly when evaluated separately on detection or localization, but not jointly on both, was affected negatively.

show abstract

A Sequence Matching Network for Polyphonic Sound Event Localization and Detection

Cited by 35 publications

References 23 publications

DCASENET: An Integrated Pretrained Deep Neural Network for Detecting and Classifying Acoustic Scenes and Events

DCASENET: An Integrated Pretrained Deep Neural Network for Detecting and Classifying Acoustic Scenes and Events

Multichannel environmental sound segmentation

Overview and Evaluation of Sound Event Localization and Detection in DCASE 2019

Contact Info

Product

Resources

About