Towards language independent acoustic modeling

Abstract

We describe procedures and experimental results using speech from diverse source languages to build an ASR system for a single target language. This work is intended to improve ASR in languages for which large amounts of training data are not available. We have developed both knowledge based and automatic methods to map phonetic units from the source languages to the target language. We employed HMM adaptation techniques and Discriminative Model Combination to combine acoustic models from the individual source languages for recognition of speech in the target language. Experiments are described in which Czech Broadcast News is transcribed using acoustic models trained from small amounts of Czech read speech augmented by English, Spanish, Russian, and Mandarin acoustic models.

Multilingual training of deep neural networks

“…Previous approaches in this direction have included: constructing a universal phone set [1,2,3]; modeling a set of universal speech attributes, such as voicing, 1 Here we do not use the term "multilingual" in the sense of codeswitching, but purely for techniques that can learn from multiple languages. nasality and frication [4]; and mapping between phones of different languages using some automated method that relies on some predefined distance measure [5,6,7] or by manually creating a mapping using acoustic phonetic knowledge [5].…”

Investigation of multilingual deep neural networks for spoken term detection

“…Traditionally researchers have either used universal phone sets [16,17,7,6] or mapped phones between languages [18,19]. Universal phone sets attempt to exploit the commonalities between the speech sounds across languages.…”

Lexical triggers and latent semantic analysis for cross-lingual language model adaptation

“…In acoustic modeling, Schultz and Waibel [1998] and Byrne et al [2000] reuse and adapt acoustic models from resource-rich languages to improve acoustic models for resourcedeficient languages. Morphological analyzers, noun-phrase chunkers, POS taggers, and so on, have been developed for resource-deficient languages by Yarowsky et al [2001] by performing linguistic analysis on the resource-rich side of a parallel or translation-equivalent corpus, and automatically projecting the annotations to the resource-deficient language via word alignment.…”