Language-independent and language-adaptive acoustic modeling for speech recognition

Abstract

With the distribution of speech technology products all over the world, the portability to new target languages becomes a practical concern. As a consequence our research focuses on the question of how to port LVCSR systems in a fast and efficient way. More specifically we want to estimate acoustic models for a new target language using speech data from varied source languages, but only limited data from the target language. For this purpose we introduce different methods for multilingual acoustic model combination and a polyphone decision tree specialization procedure. Recognition results using language dependent, independent and language adaptive acoustic models are presented and discussed in the framework of our GlobalPhone project which investigates LVCSR systems in 15 languages.

Acoustic and lexical resource constrained ASR using language-independent acoustic model and language-dependent probabilistic lexical model

“…For instance, (a) by jointly learning acoustic model parameters (GMMs) and probabilistic lexical model parameters in the framework of HMM/GMM systems as in (Luo and Jelinek, 1999), or (b) by combining acoustic model adaptation with polyphone decision tree state tying (PDTS) (Schultz and Waibel, 2001), or (c) using SGMM approach (Burget et al, 2010). Comparing probabilistic lexical modeling and deterministic lexical modeling along these lines with graphemes as subword units is part of our future work.…”

“…In the literature, lack of acoustic resources has been typically addressed using acoustic model adaptation techniques that exploit multilingual or crosslingual acoustic and lexical resources (Kohler, 1998;Beyerlein et al, 2000;Schultz and Waibel, 2001;Le and Besacier, 2009;Burget et al, 2010). Generally, the first step in these techniques is the definition of common or universal phone set across all out-of-domain languages and target language.…”

“…Multilingual acoustic model (GMMs) and lexical model (state tying) are then trained on data from out-of-domain languages. The parameters of multilingual acoustic model are adapted on target language data using techniques such as, bootstrapping, maximum a posteriori adaptation (MAP) technique, maximum likelihood linear regression (MLLR) technique, sub-space Gaussian mixture model (SGMM) approach while the out-of-domain lexical model (state tying) is either retained (Kohler, 1998;Beyerlein et al, 2000;Le and Besacier, 2009) or redefined using target language data (Schultz and Waibel, 2001;Burget et al, 2010).…”

“…Investigations until now have shown that multilingual grapheme-based ASR systems generally perform worse compared to monolingual grapheme-based ASR systems (Kanthak and Ney, 2003;Stüker, 2008a,b). This is unlike phone subword units where it has been shown that multilingual acoustic models can outperform monolingual acoustic models (Schultz and Waibel, 2001). …”

“…While this is not an issue for resource rich languages, it is challenging for under-resourced languages and domains that may not have such resources (Besacier et al, 2014). In the literature lack of acoustic resources has been typically addressed by first modeling the relationship between lexical units and feature observations on out-of-domain or language-independent data and then adapting it on target language data through techniques such as bootstrapping, maximum a posteriori adaptation (MAP) technique, maximum likelihood linear regression (MLLR) (Kohler, 1998;Beyerlein et al, 2000;Schultz and Waibel, 2001; Le and Besacier, 2009;Burget et al, 2010). The lack of phonetic lexical resources has been addressed through the use of alternate subword units, such as graphemes (Schukat-Talamazzini et al, 1993;Kanthak and Ney, 2002;Killer et al, 2003;Dines and Magimai-Doss, 2007;Ko and Mak, 2014).…”