Gated Multi-Head Attention Pooling for Weakly Labelled Audio Tagging

Abstract

Multiple instance learning (MIL) has recently been used for weakly labelled audio tagging, where the spectrogram of an audio signal is divided into segments to form instances in a bag, and then the low-dimensional features of these segments are pooled for tagging. The choice of a pooling scheme is the key to exploiting the weakly labelled data. However, the traditional pooling schemes are usually fixed and unable to distinguish the contributions, making it difficult to adapt to the characteristics of the sound events. In this paper, a novel pooling algorithm is proposed for MIL, named gated multi-head attention pooling (GMAP), which is able to attend to the information of events from different heads at different positions. Each head allows the model to learn information from different representation subspaces. Furthermore, in order to avoid the redundancy of multi-head information, a gating mechanism is used to fuse individual head features. The proposed GMAP increases the modeling power of the single-head attention with no computational overhead. Experiments are carried out on Audioset, which is a large-scale weakly labelled dataset, and show superior results to the non-adaptive pooling and the vanilla attention pooling schemes.

Multi-Attentive Detection of the Spider Monkey Whinny in the (Actual) Wild

“…Regarding embedding pooling methods, even though singlehead attention has not been shown to be better than prediction pooling [20,21], there has been a growing number of recent studies using multi-head attention on audio classification tasks [26,30,17,27]. Multiple attention heads hold the potential of learning to attend to different patterns, however, they introduce additional parameters, as well as the need to aggregate their outputs.…”

“…Alternatively, the authors of [27] propose to use a different, fixed temperature parameter per head to encourage them to attend to different event durations, and then concatenate the pooled embeddings. Finally, the authors of [17] also use concatenation of the weighted averages, as well as of the weighted standard deviation of the embeddings, as proposed in [31], before applying a gating mechanism on this aggregated vector.…”

“…Since hi,t is an embedding in a subspace of R C , we can learn a linear C × 1 energy transformation f that serves as an unnormalised proxy of the importance of each instance. In fact, recent work [26,30,17,27] has shown the value of a plural learning of such Table 2: A sketch of our proposed SE-ResNet28. The @ symbol refers to the number of filters by the respective CNN layers.…”

“…an attempt to reduce the demanding fieldwork required to detect this rare and elusive species, we used Passive acoustic monitoring (PAM) [1] by means of in situ sound recorders in order to collect an acoustic dataset, while keeping perturbations to their behaviour, by our presence, to a minimal degree. We approach the spider monkey call detection task via weakly supervised, Acoustic Event Detection (AED), specifically the Multiple Instance Learning (MIL) framework [17,18,19,20,21,22]: since our call annotations refer to the entire clip, we have to use mechanisms for pooling instance-level information to a bag-level embedding (or prediction) that characterises the entire clip.…”

“…We then perform a second comparative study, among pooling mechanisms, this time controlling the core model to be our proposed SE-ResNet28. We include in the comparison both prediction based [25,22,21], as well as embedding based [26,27,17] pooling methods. Our surprising results show that self-attention based embedding pooling is generally better than prediction pooling for our dataset, thus contradicting previous insights from similar comparisons [20,21].…”