Binary Speech Features for Keyword Spotting Tasks

Abstract

Keyword spotting is a classification task which aims to detect a specific set of spoken words. In general, this type of task runs on a power-constrained device such as a smartphone. One method to reduce the power consumption of a keyword spotting algorithm (typically a neural network) is to reduce the precision of the network weights and activations. In this paper, we propose a new representation of speech features which is more adapted to low-precision networks and compatible with binary/ternary neural networks. The new representation is based on the log-Mel spectrogram and models the variation of power over time. Tested on a ResNet, this representation produces results nearly as accurate as full-precision MFCCs, which are traditionally used in speech recognition applications.

Deep Spoken Keyword Spotting: An Overview

“…The same philosophy can be applied to speech features. Emerging research [69] studies two kinds of low-precision speech representations: linearly-quantized log-Mel spectrogram and power variation over time, derived from log-Mel spectrogram, represented by only 2 bits. Experimental results show that using 8-bit log-Mel spectra yields same KWS accuracy as employing full-precision MFCCs.…”

“…Furthermore, KWS performance degradation is insignificant when exploiting 2bit precision speech features. As the authors of [69] state, this fact might indicate that much of the spectral information is superfluous when attempting to spot a set of keywords. In [82], we independently arrived at the same finding.…”

“…Residual learning, proposed by He et al [126] for image recognition, is widely considered to implement state-of-theart acoustic models for deep KWS [30], [32], [50]- [52], [57], [67], [69], [78]. In short, residual learning models are constructed by introducing a series of shortcut connections linking non-consecutive layers (as exemplified by Figure 6), which helps to better train very deep CNN models.…”

“…Therefore, it is obvious that the non-streaming mode lacks some realism from a practical point of view. Despite this, isolated word classification is considered by a number of deep KWS works, e.g., [16], [30], [32], [48]- [52], [58], [69], [82], [89], [99], [109], [125], [128]- [130]. We believe that this is because of the simpler experimental framework with respect to that of the dynamic or streaming case.…”

“…The publicly available Google Speech Commands Dataset [153], [154] has become the de facto open benchmark for [30] for further details). Multiple recent deep KWS works have employed either the first version [16], [30], [32], [43], [48]- [52], [57], [58], [67], [69], [70], [86], [90], [100], [125] or the second version [32], [47], [48], [53], [70], [82], [89], [90], [99], [100], [109], [128]- [130], [159], [175] of the Google Speech Commands Dataset. Despite how valuable this open reference is for KWS research and development, we can raise two relevant points of criticism:…”