Dummy Prototypical Networks for Few-Shot Open-Set Keyword Spotting

Kim, Byeonggeun; Yang, Seung-Han; Chung, Inseop; Chang, Simyung

doi:10.21437/interspeech.2022-921

Cited by 9 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the context of Few-Shot learning for KWS, many works relied on the Prototypical Network (ProtoNet) approach 9 . Similarly to our proposed approach, Pro-toNet uses episodic training and prototype-based classification but cross-entropy is adopted as the optimization criteria.…”

Section: Comparison With Other Workmentioning

confidence: 99%

On-Device Customization of Tiny Deep Learning Models for Keyword Spotting With Few Examples

Rusci,

Tuytelaars

2023

IEEE Micro

View full text Add to dashboard Cite

Designing a customized KeyWord Spotting (KWS) Deep Neural Network (DNN) for tiny sensors is a time-consuming process, demanding training a new model on a remote server with a dataset of collected keywords. This paper investigates the effectiveness of a DNN-based KWS classifier that can be initialized on-device simply by recording a few examples of the target commands. At runtime, the classifier computes the distance between the DNN output and the prototypes of the recorded keywords. By experimenting with multiple TinyML models on the Google Speech Command dataset, we report an accuracy of up to 80% using only ten examples of utterances not seen during training. When deployed on a multi-core microcontroller with a power envelope of 25 mW, the most accurate ResNet15 model takes 9.7 msec to process a 1 sec speech frame, demonstrating the feasibility of on-device KWS customization for tiny devices without requiring any backpropagation-based transfer learning.A voice command sensor placed on everyday objects can recognize a set of target keywords to enable speech-controlled functionalities. The keyword classification algorithm, commonly denoted as keyword spotting (KWS) in the literature 1 , runs locally on-device to process the audio data recorded by the microphone. These smart sensors are typically battery-powered and Micro-Controllers Units (MCUs) are used as data processing engines to meet the stringent energy requirements. MCUs feature a power consumption of up to a few tens of mWs but, on the other side, present limited computation power and on-chip memory capacity, making the porting of robust speech processing pipelines on-device highly challenging.Recently, Deep Neural Networks (DNN) for Keyword Spotting have been efficiently implemented on low-power MCUs 2 . These DNN solutions feature up to only a few hundred of thousand parameters to fit the memory constraints of tiny MCU devices, typically lower than a few MBs. Small-sized solutions,

show abstract

Section: Comparison With Other Workmentioning

confidence: 99%

On-Device Customization of Tiny Deep Learning Models for Keyword Spotting With Few Examples

Rusci,

Tuytelaars

2023

IEEE Micro

View full text Add to dashboard Cite

show abstract

“…For example, Huh et al [6] explore several metric learning objectives such as triplet loss [18] and prototypical loss [19] for training KWS. In addition, Kim et al [20] suggest a multiple dummy prototype generator to handle open-set queries efficiently. Considering that KWS is closer to the detection task rather than the classification task in the real-world scenario, metric learning methods are advantageous over classification approaches, effectively tackling unknown category samples via distance metrics.…”

Section: Metric Learning Based Kwsmentioning

confidence: 99%

“…For data augmentation, we used noise corpus from [24] and RIR coefficients from [25]. The maximum value for volume scale is sampled from (0.2, 0.9) and SNR is sampled from (10,20).…”

Section: Implementation Detailsmentioning

confidence: 99%

Fully Unsupervised Training of Few-shot Keyword Spotting

Lee¹,

Kim²,

Mun³

et al. 2022

Preprint

View full text Add to dashboard Cite

For training a few-shot keyword spotting (FS-KWS) model, a large labeled dataset containing massive target keywords has known to be essential to generalize to arbitrary target keywords with only a few enrollment samples. To alleviate the expensive data collection with labeling, in this paper, we propose a novel FS-KWS system trained only on synthetic data. The proposed system is based on metric learning enabling target keywords to be detected using distance metrics. Exploiting the speech synthesis model that generates speech with pseudo phonemes instead of texts, we easily obtain a large collection of multi-view samples with the same semantics. These samples are sufficient for training, considering metric learning does not intrinsically necessitate labeled data. All of the components in our framework do not require any supervision, making our method unsupervised. Experimental results on real datasets show our proposed method is competitive even without any labeled and real datasets.

show abstract