Specialized Decision Surface and Disentangled Feature for Weakly-Supervised Polyphonic Sound Event Detection

Abstract

Sound event detection (SED) is to recognize the presence of sound events in the segment of audio and detect their onset as well as offset. SED can be regarded as a supervised learning task when strong annotations (timestamps) are available during learning. However, due to the high cost of manual strong labeling data, it becomes crucial to introduce weakly supervised learning to SED, in which only weak annotations (clip-level annotations without timestamps) are available during learning.In this paper, we approach SED as a multiple instance learning (MIL) problem and utilize a neural network framework with an embedding-level pooling module to solve it. The pooling module, which aggregates a sequence of high-level features generated by the neural network feature encoder into a single contextual feature representation, enables the model to learn with only weak annotations. We explore the self-learning ability of different pooling modules on finer information and propose a specialized decision surface (SDS) for class-wise attention pooling (cATP) module. We analyze and explained why a cATP module with SDS is better than other typical pooling modules from the perspective of feature space. According to the cooccurrence of several categories in the multi-label classification task, we also propose a disentangled feature (DF) to reduce interference between categories, which optimizes the high-level feature space by disentangling it based on class-wise identifiable information in the training set and obtaining multiple different subspaces. Experiments show that our approach achieves stateof-art performance on Task4 of the DCASE2018 challenge. On this basis,

Duration Robust Weakly Supervised Sound Event Detection

“…Wang [8] modified connectionist temporal classification (CTC) [9] in order to enable capturing long and short events effectively. An essential piece of work done in [10], analyzed the usage of attention-based high-level feature disentangling. Other studies in [11] analyzed the impact of several post-processing methods, specifically the estimation of an event-dependent threshold, in regards to WSSED.…”

An Improved Mean Teacher Based Method for Large Scale Weakly Labeled Semi-Supervised Sound Event Detection

“…Reduction rate r in the event-aware module is set to 8. For backend processing, adaptive median filter [11] is used, where the median filter size for each event is set to 1/3 of its mean duration length. For consistency, all experiments are repeated 5 times.…”

“…However, existing models generally have fixed receptive fields, which we believe are not well suited to capture the large variability in inter-and intra-event class durations, shown in Table 1. To solve this issue, post-processing methods, such as adaptive median filter [11] and double thresholding [12] were proposed. But these hand-tailored methods need prior event knowledge, which may be difficult to estimate.…”

“…Multiple branches with different kernel sizes are then fused using a softmax attention mechanism within each EA block. The student model is constructed by stacking IC-Conv-EA modules, aiming to be more event-sensitive compared to existing methods [6,7,11,12].…”

A Global-Local Attention Framework for Weakly Labelled Audio Tagging

“…There are two main MIL strategies, i.e. instance-level approach [9] and embedding-level approach [10,11]. The embedding-level approach integrates the instance-level feature representations into a bag-level contextual representation and then directly carries out bag-level classification, which shows a better performance [12].…”