Progressive Voice Trigger Detection: Accuracy vs Latency

Abstract

We present an architecture for voice trigger detection for virtual assistants. The main idea in this work is to exploit information in words that immediately follow the trigger phrase. We first demonstrate that by including more audio context after a detected trigger phrase, we can indeed get a more accurate decision. However, waiting to listen to more audio each time incurs a latency increase. Progressive Voice Trigger Detection allows us to trade-off latency and accuracy by accepting clear trigger candidates quickly, but waiting for more context to decide whether to accept more marginal examples. Using a two-stage architecture, we show that by delaying the decision for just 3% of detected true triggers in the test set, we are able to obtain a relative improvement of 66% in false rejection rate, while incurring only a negligible increase in latency.

Multi-task Learning with Cross Attention for Keyword Spotting

“…1. Block diagrams of (a) conventional multi-task learning for KWS [20,21] and (b) our proposed approach. In the conventional approach , a last layer is simply split into two branches, one for phoneme prediction and one for phrase prediction.…”

“…In the multi-task learning framework, the model is trained using both phonetic loss and phrase loss [3,20,21]. Let us assume that we sample N utterances for a mini-batch from a combined set of an ASR dataset and a KWS dataset.…”

“…Recently, multi-task learning has been applied to KWS [3,20,21] to better generalize models leveraging both large ASR and in-domain KWS datasets. In this framework, an output layer of the acoustic model is split into two branches for the two tasks.…”