Compact Feedforward Sequential Memory Networks for Small-footprint Keyword Spotting

Abstract

Due to limited resource on devices and complicated scenarios, a compact model with high precision, low computational cost and latency is expected for small-footprint keyword spotting tasks. To fulfill these requirements, in this paper, compact Feedforward Sequential Memory Network (cFSMN) which combines low-rank matrix factorization with conventional FSMN is investigated for a far-field keyword spotting task. The effect of its architecture parameters is analyzed. Towards achieving lower computational cost, multiframe prediction (MFP) is applied to cFSMN. For enhancing the modeling capacity, an advanced MFP is attempted by inserting small DNN layers before output layers. The performance is measured by area under the curve (AUC) for detection error tradeoff (DET) curves. The experiments show that compared with a well-tuned long short-term memory (LSTM) which needs the same latency and twofold computational cost, the cFSMN achieves 18.11% and 29.21% AUC relative decreases on the test sets which are recorded in quiet and noisy environment respectively. After applying advanced MFP, the system gets 0.48% and 20.04% AUC relative decrease over conventional cFSMN on the quiet and noisy test sets respectively, while the computational cost relatively reduces 46.58%.

Event-driven Pipeline for Low-latency Low-compute Keyword Spotting and Speaker Verification System

“…Here, we address the problem of keyword spotting in a more challenging setting with competing talkers. Recent studies tackle KWS for both efficient computation [4] and small footprint [6,7], while not many studies address these problems in the context of SV [3,2]. Efforts exist for either jointly solving both tasks [5] or solving a single task in the presence of background noise [10].…”

“…Our proposed model for KWS is competitive with state-of-theart models, e.g. [4,5,6]. It uses spike count frames that only incur a feature computation delay of 5 ms while more traditional methods that use log-filterbank features usually incur a feature computation delay of 30 − 40 ms.…”

“…Given that portable devices have limited memory and computational resources, algorithms that are cheap to compute and have a low memory footprint are preferred. For this reason, much effort has been focused on the use of small models that can be efficiently implemented on these devices [4,5,6,7].…”

Query-by-Example On-Device Keyword Spotting

“…They collect numerous variations of a specific keyword utterance and train neural networks (NNs) which have been promising method in the field. [1,2] have acoustic encoder and sequence matching decoder as separate modules. The NN-based acoustic models (AMs) predict senone-level Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc posteriors.…”

Audio Tagging with Compact Feedforward Sequential Memory Network and Audio-to-Audio Ratio Based Data Augmentation

“…Compared to RNN and CNN, cFSMN can achieve comparable performance while less in model parameters and more efficient in computation. Thereby, it is promoted to many other tasks, such as text to speech (TTS) [26] and smaller footprint keyword spotting (KWS) [27].…”