Joint Acoustic and Class Inference for Weakly Supervised Sound Event Detection

Abstract

Sound event detection is a challenging task, especially for scenes with multiple simultaneous events. While event classification methods tend to be fairly accurate, event localization presents additional challenges, especially when large amounts of labeled data are not available. Task4 of the 2018 DCASE challenge presents an event detection task that requires accuracy in both segmentation and recognition of events while providing only weakly labeled training data. Supervised methods can produce accurate event labels but are limited in event segmentation when training data lacks event timestamps. On the other hand, unsupervised methods that model the acoustic properties of the audio can produce accurate event boundaries but are not guided by the characteristics of event classes and sound categories. We present a hybrid approach that combines an acoustic-driven event boundary detection and a supervised label inference using a deep neural network. This framework leverages benefits of both unsupervised and supervised methodologies and takes advantage of large amounts of unlabeled data, making it ideal for large-scale weakly labeled event detection. Compared to a baseline system, the proposed approach delivers a 15% absolute improvement in F-score, demonstrating the benefits of the hybrid bottom-up, top-down approach.

Duration Robust Weakly Supervised Sound Event Detection

“…Recently, much research attention has been brought up in order to improve CRNN performance [5,6]. Kothinti [7] presented an interesting approach by separating SED into an unsupervised onset and offset estimation problem using conditional restricted Boltzmann machines (c-RBM) along with a supervised label prediction using CRNN. The results of 30% F1 development and 25 % F1 evaluation performance on the DCASE2018 task4 dataset indicate the robustness of this approach.…”

A Comprehensive Review of Polyphonic Sound Event Detection

“…This system was also found to perform rather poorly on event-based evaluations for a similar task in DCASE 2018 [106]. In addition, experiments also found that such architecture cannot be simplified and will result in poor detection accuracy [107] and thus requiring a large computational cost due to the sophisticated network.…”

“…Kothinti et al [106] approach was to develop two separate models for different purposes, as illustrated in Figure 15. The first model, which was a combination of Restricted Boltzmann Machine (RBM), conditional RBM (cRBM) and Principle Component Analysis (PCA), was in charge of the event boundary detection.…”

Sound Event Detection Utilizing Graph Laplacian Regularization with Event Co-Occurrence

“…The CNN architecture allows feature extraction robust against time and frequency shifts, which often occur in environmental sound analysis. An RNN has also been applied to SED in some works [14]- [16] to explicitly model time correlations of sound events. In particular, it has been reported that neural networks combining the CNN and a bidirectional gated recurrent unit (BiGRU) [20], [21], which can capture forward and backward temporal correlations of sound events, successfully detected sound events.…”

“…For example, a convolutional neural network (CNN)-based approach, which can detect sound events robustly against time and frequency shifts in the input acoustic feature, has been used in many works [12], [13]. Recurrent neural network (RNN)-or convolutional recurrent neural network (CRNN)-based approaches, which can capture temporal information of sound events, have also been utilized in some works [14]- [16]. These methods successfully analyze overlapping sound events with reasonable performance.…”