CaTT-KWS: A Multi-stage Customized Keyword Spotting Framework based on Cascaded Transducer-Transformer

Abstract

Most of the existing systems designed for keyword spotting (KWS) rely on a predefined set of keyword phrases. However, the ability to recognize customized keywords is crucial for tailoring interactions with intelligent devices. In this paper, we present a novel framework for customized KWS. This framework leverages the hardware-efficient LiCoNet architecture as the encoder, enhanced by a spectral-temporal pooling layer and a hybrid loss function to facilitate effective word embedding learning. The experimental results on a substantial internal dataset have demonstrated the distinct advantages of the proposed framework. LiCoNet performs at a similar level (1.98% FRR at 0.3 FAs/Hr) to the computationally intensive Conformer, which requires 13x computational resources.

The NPU-ASLP System for The ISCSLP 2022 Magichub Code-Swiching ASR Challenge

“…Recently, there has been increasing interest in unifying multi-stage modules into one single model. In this direction, Cascaded Transducer-Transformer (CATT-KWS) uses two-pass models, which unify streaming and non-streaming ASR approaches [19,20], to unify multistage KWS into one model [21]. Specifically, it uses the streaming part, which is originally used to generate streaming hypotheses, as the first-stage model to detect possible keywords, and then uses the non-streaming parts, which are originally used to re-score streaming hypotheses, as the validation stages for further verification of keywords detected in the first stage.…”