A 23-<i>μ</i>W Keyword Spotting IC With Ring-Oscillator-Based Time-Domain Feature Extraction

Kim, Kwantae; Gao, Chang; Graça, Rui; Kiselev, Ilya; Yoo, Hoi‐Jun; Delbrück, Tobi; Liu, Shih-Chii

doi:10.1109/jssc.2022.3195610

Cited by 16 publications

(7 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An attractive application of DNs is for online incremental training, where new labeled data become available in the field to an edge device and must be incorporated into the RNN to personalize or improve accuracy. To evaluate the performance of Delta RNNs on CIL tasks, we use GSCD v2 [23], a dataset frequently used for benchmarking ASIC keyword spotting implementations [12,20,8]. It contains 105,829 utterances of 35 English words.…”

Section: Incremental Keyword Learning Experimentsmentioning

confidence: 99%

Exploiting Symmetric Temporally Sparse BPTT for Efficient RNN Training

Chen,

Gao,

Wang

et al. 2024

AAAI

View full text Add to dashboard Cite

Recurrent Neural Networks (RNNs) are useful in temporal sequence tasks. However, training RNNs involves dense matrix multiplications which require hardware that can support a large number of arithmetic operations and memory accesses. Implementing online training of RNNs on the edge calls for optimized algorithms for an efficient deployment on hardware. Inspired by the spiking neuron model, the Delta RNN exploits temporal sparsity during inference by skipping over the update of hidden states from those inactivated neurons whose change of activation across two timesteps is below a defined threshold. This work describes a training algorithm for Delta RNNs that exploits temporal sparsity in the backward propagation phase to reduce computational requirements for training on the edge. Due to the symmetric computation graphs of forward and backward propagation during training, the gradient computation of inactivated neurons can be skipped. Results show a reduction of ∼80% in matrix operations for training a 56k parameter Delta LSTM on the Fluent Speech Commands dataset with negligible accuracy loss. Logic simulations of a hardware accelerator designed for the training algorithm show 2-10X speedup in matrix computations for an activation sparsity range of 50%-90%. Additionally, we show that the proposed Delta RNN training will be useful for online incremental learning on edge devices with limited computing resources.

show abstract

Section: Incremental Keyword Learning Experimentsmentioning

confidence: 99%

Exploiting Symmetric Temporally Sparse BPTT for Efficient RNN Training

Chen,

Gao,

Wang

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Kim proposed a dedicated chip for wearables and IoT devices to use keyword spotting models. [24]. However, these studies on small-footprint keyword spotting do not consider noise robustness that is crucial for a successful voice interface.…”

Section: A Small-footprint Abstractpottingmentioning

confidence: 99%

“…Unlike smart speakers with high-performance processors [21], wearables and IoT devices utilize low-performance and low-power microcontrollers [22] for spoken keyword spotting. Although smart speakers show good performance using multiple microphones [23], it is difficult to mount multiple microphones due to the nature of those small devices [24]. Therefore, developing a noise-robust and small-footprint keyword spotting for the devices with minimal resources and a single microphone is required.…”

Section: Introductionmentioning

confidence: 99%

Joint Framework of Curriculum Learning and Knowledge Distillation for Noise-Robust and Small-Footprint Keyword Spotting

Lim,

Baek

2023

IEEE Access

View full text Add to dashboard Cite

Spoken keyword spotting has been widely used to facilitate an always-on voice interface in consumer electronics, owing to its simplicity and low latency. Small-footprint keyword spotting based on tiny convolutional neural networks can be implemented on resource-constrained but energy-efficient microcontrollers in real time. However, it is difficult for tiny neural networks to learn the noise-robustness properties essential for successful voice interfaces. To overcome this problem, this study proposes a joint framework of curriculum learning and knowledge distillation for noise-robust small-footprint keyword spotting. The proposed joint framework applies noise mixture curriculum learning to a network that is sufficiently large for learning various noise situations. Subsequently, knowledge distillation is applied to compress the large network into a network small enough to be onboard microcontrollers. To enhance the effectiveness of the joint framework, we propose curriculum learning using a new noise mixture strategy and knowledge distillation with an effective ensemble of neural network snapshots for each curriculum stage. These proposed methods enable the effective learning of noisy situations by large networks and the transfer of noise robustness to small networks. The effectiveness of our joint framework is illustrated using the Google Speech Commands dataset with noise mixtures from various public noise datasets. Our joint framework results in superior performance in noisy situations compared with state-of-the-art noise-robust keyword spotting methods. Therefore, the proposed framework significantly improves the usability of voice interfaces in consumer electronics.

show abstract

“…Using such VCO-based filters, an efficient feature extraction architecture was proposed in [8] and implemented in [9], [10]. Compared to analog solutions, the VCO-based approach is free of area consuming elements (e.g.…”

Section: Introductionmentioning

confidence: 99%

Analysis of in-band Spurious Tones of VCO-based Analog Filters and Mitigation Techniques

Medina

Loi

Rombouts

et al. 2023

2023 IEEE International Symposium on Circuits and Systems (ISCAS)

View full text Add to dashboard Cite

Recently VCO-based filters have been used to implement feature extraction in voice recognition systems. Typically implemented filter structures embed the VCOs in a feedback loop. In these implementations, harmonic distortion has been observed in the output signal. This harmonic distortion has been attributed to ring oscillator nonlinearity but its origin stems from a system level problem. In this manuscript we will analyze this problem by looking at the cross modulation of Pulse Frequency Modulation (PFM) side-bands and we will explain how this creates the previously misunderstood harmonic distortion. Moreover, we will propose efficient techniques to mitigate these harmonic components and show that they can easily be implemented in hardware.

show abstract

A 23-μW Keyword Spotting IC With Ring-Oscillator-Based Time-Domain Feature Extraction

Cited by 16 publications

References 39 publications

Exploiting Symmetric Temporally Sparse BPTT for Efficient RNN Training

Exploiting Symmetric Temporally Sparse BPTT for Efficient RNN Training

Joint Framework of Curriculum Learning and Knowledge Distillation for Noise-Robust and Small-Footprint Keyword Spotting

Analysis of in-band Spurious Tones of VCO-based Analog Filters and Mitigation Techniques

Contact Info

Product

Resources

About