Few-Shot Keyword Spotting in Any Language

Mazumder, Mark; Banbury, Colby; Meyer, Josh; Warden, Pete; Reddi, Vijay Janapa

doi:10.21437/interspeech.2021-1966

Cited by 16 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The feature extractor is trained on a source dataset disjoined from the target scenario. We employ the recent Multilingual Spoken Words Corpus (MSWC) dataset 6 , which includes up to 39k unique utterances in the English train partition. The average number of occurrences per class is 180; class distribution is highly unbalanced.…”

Section: Dnn Feature Extractor Designmentioning

confidence: 99%

See 1 more Smart Citation

On-Device Customization of Tiny Deep Learning Models for Keyword Spotting With Few Examples

Rusci,

Tuytelaars

2023

IEEE Micro

View full text Add to dashboard Cite

Designing a customized KeyWord Spotting (KWS) Deep Neural Network (DNN) for tiny sensors is a time-consuming process, demanding training a new model on a remote server with a dataset of collected keywords. This paper investigates the effectiveness of a DNN-based KWS classifier that can be initialized on-device simply by recording a few examples of the target commands. At runtime, the classifier computes the distance between the DNN output and the prototypes of the recorded keywords. By experimenting with multiple TinyML models on the Google Speech Command dataset, we report an accuracy of up to 80% using only ten examples of utterances not seen during training. When deployed on a multi-core microcontroller with a power envelope of 25 mW, the most accurate ResNet15 model takes 9.7 msec to process a 1 sec speech frame, demonstrating the feasibility of on-device KWS customization for tiny devices without requiring any backpropagation-based transfer learning.A voice command sensor placed on everyday objects can recognize a set of target keywords to enable speech-controlled functionalities. The keyword classification algorithm, commonly denoted as keyword spotting (KWS) in the literature 1 , runs locally on-device to process the audio data recorded by the microphone. These smart sensors are typically battery-powered and Micro-Controllers Units (MCUs) are used as data processing engines to meet the stringent energy requirements. MCUs feature a power consumption of up to a few tens of mWs but, on the other side, present limited computation power and on-chip memory capacity, making the porting of robust speech processing pipelines on-device highly challenging.Recently, Deep Neural Networks (DNN) for Keyword Spotting have been efficiently implemented on low-power MCUs 2 . These DNN solutions feature up to only a few hundred of thousand parameters to fit the memory constraints of tiny MCU devices, typically lower than a few MBs. Small-sized solutions,

show abstract

Section: Dnn Feature Extractor Designmentioning

confidence: 99%

“…Recently, Mazumder et al 6 used transfer learning to train a DNN-based classifier with few shots. This approach places a linear classifier on top of a backbone model trained on the multilingual MSWC dataset.…”

Section: Comparison With Other Workmentioning

confidence: 99%

On-Device Customization of Tiny Deep Learning Models for Keyword Spotting With Few Examples

Rusci,

Tuytelaars

2023

IEEE Micro

View full text Add to dashboard Cite

show abstract

“…their encoder and show better generalization ability under fewshot learning. Mazumder et al [10] train a multi-class multilignual keyword classification model with EfficientNet's structure [11] as the encoder on Common Voice [12] to solve multilingual few-shot KWS problem. Nonetheless, preparing largescale KWS datasets usually requires audios, transcription, and a forced aligner, which increases the cost.…”

Section: Train a Multi-class Keyword Classification Model On Librispe...mentioning

confidence: 99%

On the Efficiency of Integrating Self-supervised Learning and Meta-learning for User-defined Few-shot Keyword Spotting

Kao¹,

Wu²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

User-defined keyword spotting is a task to detect new spoken terms defined by users. This can be viewed as a few-shot learning problem since it is unreasonable for users to define their desired keywords by providing many examples. To solve this problem, previous works try to incorporate self-supervised learning models or apply meta-learning algorithms. But it is unclear whether self-supervised learning and meta-learning are complementary and which combination of the two types of approaches is most effective for few-shot keyword discovery. In this work, we systematically study these questions by utilizing various self-supervised learning models and combining them with a wide variety of meta-learning algorithms. Our result shows that HuBERT combined with Matching network achieves the best result and is robust to the changes of few-shot examples.

show abstract

“…Such classification layers can be independent of each other and use the same embedding model, since in the adaptation phase only the last layer is modified. A similar solution based on a few-shot transfer learning is described in [16]. It should be noted that usually KWS solutions are deployed on devices with limited resources, hence performing any type of model adaptation might be troublesome.…”

Section: Introductionmentioning

confidence: 99%

Open Vocabulary Keyword Spotting with Small-Footprint ASR-based Architecture and Language Models

Pudo,

Wosik,

Janicki

2023

Annals of Computer Science and Information Systems

View full text Add to dashboard Cite

We present the results of experiments on minimizing the model size for the text-based Open Vocabulary Keyword Spotting task. The main goal is to perform inference on devices with limited computing power, such as mobile phones. Our solution is based on the acoustic model architecture adopted from the automatic speech recognition task. We extend the acoustic model with a simple yet powerful language model, which improves recognition results without impacting latency and memory footprint. We also present a method to improve the recognition rate of rare keywords based on the recordings generated by a text-to-speech system. Evaluations using a public testset prove that our solution can achieve a true positive rate in the range of 73%-86%, with a false positive rate below 24%. The model size is only 3.2 MB, and the real-time factor measured on contemporary mobile phones is 0.05.

show abstract

Few-Shot Keyword Spotting in Any Language

Cited by 16 publications

References 11 publications

On-Device Customization of Tiny Deep Learning Models for Keyword Spotting With Few Examples

On-Device Customization of Tiny Deep Learning Models for Keyword Spotting With Few Examples

On the Efficiency of Integrating Self-supervised Learning and Meta-learning for User-defined Few-shot Keyword Spotting

Open Vocabulary Keyword Spotting with Small-Footprint ASR-based Architecture and Language Models

Contact Info

Product

Resources

About