Unsupervised Contrastive Learning of Sound Event Representations

Abstract

Self-supervised representation learning can mitigate the limitations in recognition tasks with few manually labeled data but abundant unlabeled data-a common scenario in sound event research. In this work, we explore unsupervised contrastive learning as a way to learn sound event representations. To this end, we propose to use the pretext task of contrasting differently augmented views of sound events. The views are computed primarily via mixing of training examples with unrelated backgrounds, followed by other data augmentations. We analyze the main components of our method via ablation experiments. We evaluate the learned representations using linear evaluation, and in two in-domain downstream sound event classification tasks, namely, using limited manually labeled data, and using noisy labeled data. Our results suggest that unsupervised contrastive pre-training can mitigate the impact of data scarcity and increase robustness against noisy labels.

Improving Sound Event Classification by Increasing Shift Invariance in Convolutional Neural Networks

“…We choose mixup because the concept of mixing sounds is an audio-informed operation, and it has been proven useful for SET [2,3,29] and other sound event research tasks [30]. In our view, mixup can be interpreted from two different perspectives.…”

Self-Supervised Learning from Automatically Separated Sound Scenes

“…This paradigm has seen major progress in computer vision [9,10,11] and in speech recognition [12,13,7]. For general-purpose audio, including a variety of environmental sounds beyond speech, the majority of works are based on contrastive learning [14,15,16,17,18,19], where a representation is learned by comparing pairs of examples selected by some semantically-correlated notion of similarity [20]. Specifically, comparisons are made between positive pairs of "similar" and negative pairs of "dissimilar" examples, with the goal of learning a representation that pulls together positive pairs and thus reflects semantic structure.…”

“…Recently, promising results have been attained by contrastive learning approaches that solve the proxy task of similarity maximization [17,18,19], following the seminal SimCLR work in visual representation learning [9]. This method consists of maximizing the similarity between differently-augmented views of the same input audio example.…”

“…This method consists of maximizing the similarity between differently-augmented views of the same input audio example. Critical to its success is the simultaneous use of a diversity of semantics-preserving, domain-specific augmentation methods [9,18]. For audio modeling, proven augmentation strategies include sampling nearby audio frames [14,15,17,18,19], artificial example mixing [18,19,14], time/frequency masking [18,19], random resized cropping [18] and time/frequency shifts [14,18,19].…”

“…Critical to its success is the simultaneous use of a diversity of semantics-preserving, domain-specific augmentation methods [9,18]. For audio modeling, proven augmentation strategies include sampling nearby audio frames [14,15,17,18,19], artificial example mixing [18,19,14], time/frequency masking [18,19], random resized cropping [18] and time/frequency shifts [14,18,19]. In most cases, these augmentations introduce artificial, handcrafted transformations with hyperparameters that must be tuned to lie within a semantics-preserving range.…”