RNN-LDA Clustering for Feature Based DNN Adaptation

Abstract

Model based deep neural network (DNN) adaptation approaches often require multi-pass decoding in test time. Input feature based DNN adaptation, for example, based on latent Dirichlet allocation (LDA) clustering, provide a more efficient alternative. In conventional LDA clustering, the transition and correlation between neighboring clusters is ignored. In order to address this issue, a recurrent neural network (RNN) based clustering scheme is proposed to learn both the standard LDA cluster labels and their natural correlation over time in this paper. In addition to directly using the resulting RNN-LDA as input features during DNN adaptation, a range of techniques were investigated to condition the DNN hidden layer parameters or activation outputs on the RNN-LDA features. On a DARPA Gale Mandarin Chinese broadcast speech transcription task, the proposed RNN-LDA cluster features adapted DNN system outperformed both the baseline un-adapted DNN system and conventional LDA features adapted DNN system by 8% relative on the most difficult Phoenix TV subset. Consistent improvements were also obtained after further combination with model based adaptation approaches.

Exploiting Speaker and Phonetic Diversity of Mismatched Language Resources for Unsupervised Subword Modeling

“…It is widely acknowledged that speaker adaptive training (SAT) is effective in improving ASR performance, especially for large vocabulary tasks [18][19][20]. Approaches to SAT are divided into two categories: model-based approaches, e.g., maximum likelihood linear regression (MLLR) [21], and feature-based approaches, e.g., feature-space MLLR (fMLLR) [22], i-vectors [23], speaker codes [24], and other appending features [25]. All of these methods are based on the assumption that speech transcription and/or speaker identity information are available.…”