A Comparison of Five Multiple Instance Learning Pooling Functions for Sound Event Detection with Weak Labeling

Abstract

Sound event detection (SED) entails two subtasks: recognizing what types of sound events are present in an audio stream (audio tagging), and pinpointing their onset and offset times (localization). In the popular multiple instance learning (MIL) framework for SED with weak labeling, an important component is the pooling function. This paper compares five types of pooling functions both theoretically and experimentally, with special focus on their performance of localization. Although the attention pooling function is currently receiving the most attention, we find the linear softmax pooling function to perform the best among the five. Using this pooling function, we build a neural network called TALNet. It is the first system to reach state-of-the-art audio tagging performance on Audio Set, while exhibiting strong localization performance on the DCASE 2017 challenge at the same time.

A Comparison of Pooling Methods on LSTM Models for Rare Acoustic Event Classification

“…Recent research investigates memory dynamics and control in recurrent neural networks (RNNs) including LSTM [22]. There are comparisons on different pooling functions for AEC/AED [9], and spatio-temporal attention pooling proposed for audio scene classification [23].…”

“…Wang et al [9] did a thorough analysis theoretically and experimentally of five pooling functions on prediction. The analysis was done for multiple instance learning framework on AED with weak labeling, whose goal is to detect and localize events at the same time.…”

“…This dataset employs a subset of AudioSet [25], which includes 10-second clips containing 17 sound events from two categories: "Warning" and "Vehicle". Although audio tagging, which is similar to our utterance-level classification task, is covered in the experiments in [9], there are two fundamental differences between this work and theirs. 1) Different characteristics of events: Our work focuses on rare events, and we conducted experiments on DCASE 2017 task 2 [24]: detection of rare sound events.…”

“…2) Different focuses of analysis: In [9], the analysis focuses on the effect of pooling functions on both event localization and classification for weak labeling. Due to the requirement of event localization, only pooling functions on prediction were discussed in [9]. Our work analyzes the effect of pooling functions on LSTM based AEC mod-…”

“…Experiments are designed for understanding the memory dynamics for LSTM models, and looking for solutions to mitigate the sensitivity to event positions. Moreover, we further discuss four pooling functions on the feature side, which are not covered in [9]. In this paper, we investigate the dynamics of LSTM memory on AEC tasks, including an analysis on LSTM memory retaining, and a benchmarking for impacts of different pooling approaches on LSTM memory dynamics and AEC accuracies, using 1.7M synthesized clips (across 3 event types and 3 SNRs).…”