Improving Factored Hybrid HMM Acoustic Modeling without State Tying

Abstract

In this work, we show that a factored hybrid hidden Markov model (FH-HMM) which is defined without any phonetic state-tying outperforms a state-of-the-art hybrid HMM. The factored hybrid HMM provides a link to transducer models in the way it models phonetic (label) context while preserving the strict separation of acoustic and language model of the hybrid HMM approach. Furthermore, we show that the factored hybrid model can be trained from scratch without using phonetic state-tying in any of the training steps. Our modeling approach enables triphone context while avoiding phonetic state-tying by a decomposition into locally normalized factored posteriors for monophones/HMM states in phoneme context. Experimental results are provided for Switchboard 300h and LibriSpeech. On the former task we also show that by avoiding the phonetic state-tying step, the factored hybrid can take better advantage of regularization techniques during training, compared to the standard hybrid HMM with phonetic state-tying based on classification and regression trees (CART).

Principled Comparisons for End-to-End Speech Recognition: Attention vs Hybrid at the 1000-Hour Scale

“…We develop a multi-head decoding method, which yields the best results. We find that framelevel training is still useful, but on the other hand, expert pronunciation-lexicons and tree-clustering for state-tying do not appear necessary, echoing other recent work [14], [15], and potentially simplifying the HMM / DNN-system.…”

“…Additionally, we find that the simple pruning Flat Start LF-MMI outperforms the treeclustering state-tied LF-MMI in our test on Finnish Parliament Train20. This is surprising, but seems to suggest that the treeclustering does not always yield better performance in HMM acoustic modeling, which is also suggested in other recent work [15] and would simplify the HMM / DNN-system further. The investigation of this phenomenon is out of scope for this work.…”