Multi-Branch Learning for Weakly-Labeled Sound Event Detection

Huang, Yuxin; Wang, Xiangdong; Lin, Liwei; Liu, Hong; Ye, Qian

doi:10.1109/icassp40776.2020.9053023

Cited by 11 publications

(6 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, though different pooling strategies have been proposed, the correlation between the pooling methods and the classifier backends might be under investigation. Attention level pooling methods seem to be preferred for models without sequential model capabilities [6], [13] (e.g., CNN), whereas max and linear softmax pooling functions have seen success in CRNN frameworks [11], [5].…”

Section: Related Workmentioning

confidence: 99%

“…Moreover, it is indicated that feature-level aggregation methods (e.g., hidden layer) should be preferred over event-level methods (e.g., output layer). Recent work in [13] proposes multibranch learning, similar to multi-task learning, which utilizes a multitude of temporal pooling strategies in order to prevent overfitting towards a single method. Similar methods regarding disentanglement are spectral event-specific masking, as introduced in [14].…”

Section: Related Workmentioning

confidence: 99%

“…It is notable that previously well-performing models [6], [13], [17], [38] all require a fully-labeled development dataset in order to estimate a per event median filter. By contrast, our approach does not rely on such labels yet still achieves competitive results.…”

Section: Dcase2018mentioning

confidence: 99%

See 2 more Smart Citations

Towards Duration Robust Weakly Supervised Sound Event Detection

Dinkel

2021

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Sound event detection (SED) is the task of tagging the absence or presence of audio events and their corresponding interval within a given audio clip. While SED can be done using supervised machine learning, where training data is fully labeled with access to per event timestamps and duration, our work focuses on weakly-supervised sound event detection (WSSED), where prior knowledge about an event's duration is unavailable. Recent research within the field focuses on improving segment-and event-level localization performance for specific datasets regarding specific evaluation metrics. Specifically, wellperforming event-level localization work requires fully labeled development subsets to obtain event duration estimates, which significantly benefits localization performance. Moreover, wellperforming segment-level localization models output predictions at a coarse-scale (e.g., 1 second), hindering their deployment on datasets containing very short events (< 1 second). This work proposes a duration robust CRNN (CDur) framework, which aims to achieve competitive performance in terms of segmentand event-level localization. In the meantime, this paper proposes a new post-processing strategy named "Triple Threshold" and investigates two data augmentation methods along with a label smoothing method within the scope of WSSED. Event-, Segment-, and Tagging-F1 scores are utilized as primary evaluation metrics. Evaluation of our model is done on three publicly available datasets: Task 4 of the DCASE2017 and 2018 datasets, as well as URBAN-SED. Our model outperforms other approaches on the DCASE2018 and URBAN-SED datasets without requiring prior duration knowledge. In particular, our model is capable of similar performance to strongly-labeled supervised models on the URBAN-SED dataset. Lastly, we run a series of ablation experiments to reveal that without post-processing, our model's localization performance drop is significantly lower compared with other approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Towards Duration Robust Weakly Supervised Sound Event Detection

Dinkel

2021

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…To liberate model from such trade-off, Lin et al [29] propose a teacher-student framework named Guided Learning where teacher model focuses on the audio tagging, while student model focuses on the event boundaries. In addition, Huang et al [30] regard weakly supervised SED as a kind of multi-task learning and propose a multi-branch learning method to ensure the encoder can capture more comprehensive feature fit for various subtasks. Synthetic data is another alternative to train SED models, which is generated by mixing up isolated events with a background noise randomly.…”

Section: B Sound Event Detection With Weakly-labeled and Synthetic Datamentioning

confidence: 99%

Sound Event Detection Transformer: An Event-based End-to-End Model for Sound Event Detection

Ye¹,

Wang²,

Liu³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Sound event detection (SED) has gained increasing attention with its wide application in surveillance, video indexing, etc. Existing models in SED mainly generate frame-level predictions, converting it into a sequence multi-label classification problem, which inevitably brings a trade-off between event boundary detection and audio tagging when using weakly labeled data to train the model. Besides, it needs post-processing and cannot be trained in an end-to-end way. This paper firstly presents the 1D Detection Transformer (1D-DETR), inspired by Detection Transformer. Furthermore, given the characteristics of SED, the audio query and a one-to-many matching strategy for fine-tuning the model are added to 1D-DETR to form the model of Sound Event Detection Transformer (SEDT), which generates event-level predictions, end-to-end detection. Experiments are conducted on the URBAN-SED dataset and the DCASE2019 Task4 dataset, and both experiments have achieved competitive results compared with SOTA models. The application of SEDT on SED shows that it can be used as a framework for onedimensional signal detection and may be extended to other similar tasks.

show abstract

“…Thanks to the development of deep learning approaches, recent advances [4,5] have led to improved performance in SED task. Several standard convolutional neural network (CNN) blocks were stacked as the feature encoder to generate the high-level feature representations for the SED task [6,7]. Lu et al [8] proposed a multi-scale recurrent neural network (RNN) to capture the fine-grained and longterm dependencies of sound events.…”

Section: Introductionmentioning

confidence: 99%

A Multi-grained based Attention Network for Semi-supervised Sound Event Detection

Hu,

Zhu,

et al. 2022

Preprint

View full text Add to dashboard Cite

Sound event detection (SED) is an interesting but challenging task due to the scarcity of data and diverse sound events in real life. This paper presents a multi-grained based attention network (MGA-Net) for semi-supervised sound event detection. To obtain the feature representations related to sound events, a residual hybrid convolution (RH-Conv) block is designed to boost the vanilla convolution's ability to extract the time-frequency features. Moreover, a multi-grained attention (MGA) module is designed to learn temporal resolution features from coarse-level to fine-level. With the MGA module, the network could capture the characteristics of target events with short-or long-duration, resulting in more accurately determining the onset and offset of sound events. Furthermore, to effectively boost the performance of the Mean Teacher (MT) method, a spatial shift (SS) module as a data perturbation mechanism is introduced to increase the diversity of data. Experimental results show that the MGA-Net outperforms the published state-of-the-art competitors, achieving 53.27% and 56.96 % event-based macro F1 (EB-F1) score, 0.709 and 0.739 polyphonic sound detection score (PSDS) on the validation and public set respectively.

show abstract

Multi-Branch Learning for Weakly-Labeled Sound Event Detection

Cited by 11 publications

References 21 publications

Towards Duration Robust Weakly Supervised Sound Event Detection

Towards Duration Robust Weakly Supervised Sound Event Detection

Sound Event Detection Transformer: An Event-based End-to-End Model for Sound Event Detection

A Multi-grained based Attention Network for Semi-supervised Sound Event Detection

Contact Info

Product

Resources

About