Multi-label Few-shot Learning for Sound Event Recognition

Abstract

Recognizing multiple labels of an image is a practical yet challenging task, and remarkable progress has been achieved by searching for semantic regions and exploiting label dependencies. However, current works utilize RNN/LSTM to implicitly capture sequential region/label dependencies, which cannot fully explore mutual interactions among the semantic regions/labels and do not explicitly integrate label co-occurrences. In addition, these works require large amounts of training samples for each category, and they are unable to generalize to novel categories with limited samples. To address these issues, we propose a knowledge-guided graph routing (KGGR) framework, which unifies prior knowledge of statistical label correlations with deep neural networks. The framework exploits prior knowledge to guide adaptive information propagation among different categories to facilitate multi-label analysis and reduce the dependency of training samples. Specifically, it first builds a structured knowledge graph to correlate different labels based on statistical label co-occurrence. Then, it introduces the label semantics to guide learning semantic-specific features to initialize the graph, and it exploits a graph propagation network to explore graph node interactions, enabling learning contextualized image feature representations. Moreover, we initialize each graph node with the classifier weights for the corresponding label and apply another propagation network to transfer node messages through the graph. In this way, it can facilitate exploiting the information of correlated labels to help train better classifiers, especially for labels with limited training samples. We conduct extensive experiments on the traditional multi-label image recognition (MLR) and multi-label few-shot learning (ML-FSL) tasks and show that our KGGR framework outperforms the current state-of-the-art methods by sizable margins on the public benchmarks.

MetaAudio: A Few-Shot Audio Classification Benchmark

“…Few-Shot Acoustics Currently only a handful of studies exist that look at either few-shot audio classification or event detection. Of these, two are set in event detection [7,6] (classification of parts of an audio clip in time) with the other two focused on classification [14,15] (classification of an entire audio clip), the focus of this work. Comparing these works, we see a variety of approaches taken toward dataset processing, split formulation and reproducibility.…”

“…Acoustic classification and event detection have been well studied in conventional fully supervised machine learning [4,5], with many public datasets having a common evaluation protocol that is adhered to by the community, allowing for standardisation and fair comparison. This has however not extended to the few-shot equivalent, where the majority of the works that do exist make little attempt at preserving reproducibility, typically with respect to dataset management and lack of public source code [6,7]. This absence of standardisation poses significant issues when looking to compare novel and existing methods alike.…”

FSD50K: An Open Dataset of Human-Labeled Sound Events

“…The vocabulary used so far comprises 395 classes, yet many of them have few data (few tens of clips). While they are not adequate for common machine learning standards (e.g., deep learning), they can be useful for other practices requiring less data (e.g., few shot learning [72]). Likewise, this information can provide insight as to the specific content of the dataset.…”

Learning from Very Few Samples: A Survey

“…information extraction [211], matchine translation [100], charge prediction [178], sequence labeling [349] Audio&Speech audio/speech/sound classification [350], [351], [352], [353], [354], [355], text-to-speech [356], [357], [358], [359], acoustic/sound event detection [360], [361], [362], speech generation [350], [363], keyword/command recognition [364], keyword spotting [365], human-fall detection [366], speaker recognition [367],…”