Weakly-supervised audio event detection using event-specific Gaussian filters and fully convolutional networks

Su, Tingwei; Liu, Jen-Yu; Yang, Yi-Hsuan

doi:10.1109/icassp.2017.7952264

Cited by 49 publications

(38 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The choice of the pooling function is an important decision. The default choice is the "max" pooling function [12,13], which is faithful to the SMI assumption. A previous study of ours [14] has evaluated a "noisy-or" pooling function [15,16,17], and has shown it does not work for localization despite its nice probabilistic interpretation under the SMI assumption.…”

Section: Introductionmentioning

confidence: 99%

A Comparison of Five Multiple Instance Learning Pooling Functions for Sound Event Detection with Weak Labeling

Wang

Metze

2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

147

136

View full text Add to dashboard Cite

Sound event detection (SED) entails two subtasks: recognizing what types of sound events are present in an audio stream (audio tagging), and pinpointing their onset and offset times (localization). In the popular multiple instance learning (MIL) framework for SED with weak labeling, an important component is the pooling function. This paper compares five types of pooling functions both theoretically and experimentally, with special focus on their performance of localization. Although the attention pooling function is currently receiving the most attention, we find the linear softmax pooling function to perform the best among the five. Using this pooling function, we build a neural network called TALNet. It is the first system to reach state-of-the-art audio tagging performance on Audio Set, while exhibiting strong localization performance on the DCASE 2017 challenge at the same time.

show abstract

Section: Introductionmentioning

confidence: 99%

A Comparison of Five Multiple Instance Learning Pooling Functions for Sound Event Detection with Weak Labeling

Wang

Metze

2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

147

136

View full text Add to dashboard Cite

show abstract

“…In [30], a mel spectrogram of an audio clip is presented to a CNN, where the filters of each convolutional layer capture local patterns of a spectrogram. After a global pooling layer such as global max pooling [28], global average pooling [31], global weighted rank pooling [23], global attention pooling [32,33] or other poolings [34,35], fully connected layers are applied to predict the presence probabilities of audio classes. Fig.…”

Section: B Convolutional Neural Network For Audio Tagging and Weaklmentioning

confidence: 99%

Sound Event Detection and Time–Frequency Segmentation from Weakly Labelled Data

Kong

Sobieraj

et al. 2019

IEEE/ACM Trans. Audio Speech Lang. Process.

101

View full text Add to dashboard Cite

Sound event detection (SED) aims to detect when and recognize what sound events happen in an audio clip. Many supervised SED algorithms rely on strongly labelled data which contains the onset and offset annotations of sound events. However, many audio tagging datasets are weakly labelled, that is, only the presence of the sound events is known, without knowing their onset and offset annotations. In this paper, we propose a time-frequency (T-F) segmentation framework trained on weakly labelled data to tackle the sound event detection and separation problem. In training, a segmentation mapping is applied on a T-F representation, such as log mel spectrogram of an audio clip to obtain T-F segmentation masks of sound events.The T-F segmentation masks can be used for separating the sound events from the background scenes in the time-frequency domain. Then a classification mapping is applied on the T-F segmentation masks to estimate the presence probabilities of the sound events. We model the segmentation mapping using a convolutional neural network and the classification mapping using a global weighted rank pooling (GWRP). In SED, predicted onset and offset times can be obtained from the T-F segmentation masks. As a byproduct, separated waveforms of sound events can be obtained from the T-F segmentation masks. We remixed the DCASE 2018 Task 1 acoustic scene data with the DCASE 2018 Task 2 sound events data. When mixing under 0 dB, the proposed method achieved F1 scores of 0.534, 0.398 and 0.167 in audio tagging, frame-wise SED and event-wise SED, outperforming the fully connected deep neural network baseline of 0.331, 0.237 and 0.120, respectively. In T-F segmentation, we achieved an F1 score of 0.218, where previous methods were not able to do T-F segmentation.

show abstract

“…Understanding the surrounding environment through sounds has been considered as a major component in many daily applications, such as surveillance, smart city, and smart cars [1,2]. Recent years have witnessed great progress in sound event detection and classification using deep learning techniques [3][4][5][6][7]. However, most prior arts rely on standard supervised learning algorithm and may not perform well for sound events with sparse training examples.…”

Section: Introductionmentioning

confidence: 99%

Learning to Match Transient Sound Events Using Attentional Similarity for Few-shot Sound Recognition

Chou

Cheng

Jang

et al. 2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

In this paper, we introduce a novel attentional similarity module for the problem of few-shot sound recognition. Given a few examples of an unseen sound event, a classifier must be quickly adapted to recognize the new sound event without much fine-tuning. The proposed attentional similarity module can be plugged into any metric-based learning method for few-shot learning, allowing the resulting model to especially match related short sound events. Extensive experiments on two datasets show that the proposed module consistently improves the performance of five different metricbased learning methods for few-shot sound recognition. The relative improvement ranges from +4.1% to +7.7% for 5-shot 5-way accuracy for the ESC-50 dataset, and from +2.1% to +6.5% for noiseESC-50. Qualitative results demonstrate that our method contributes in particular to the recognition of transient sound events.

show abstract

Weakly-supervised audio event detection using event-specific Gaussian filters and fully convolutional networks

Cited by 49 publications

References 14 publications

A Comparison of Five Multiple Instance Learning Pooling Functions for Sound Event Detection with Weak Labeling

A Comparison of Five Multiple Instance Learning Pooling Functions for Sound Event Detection with Weak Labeling

Sound Event Detection and Time–Frequency Segmentation from Weakly Labelled Data

Learning to Match Transient Sound Events Using Attentional Similarity for Few-shot Sound Recognition

Contact Info

Product

Resources

About