Universal Phone Recognition with a Multilingual Allophone System

Abstract

Multilingual models can improve language processing, particularly for low resource situations, by sharing parameters across languages. Multilingual acoustic models, however, generally ignore the difference between phonemes (sounds that can support lexical contrasts in a particular language) and their corresponding phones (the sounds that are actually spoken, which are language independent). This can lead to performance degradation when combining a variety of training languages, as identically annotated phonemes can actually correspond to several different underlying phonetic realizations. In this work, we propose a joint model of both language-independent phone and language-dependent phoneme distributions. In multilingual ASR experiments over 11 languages, we find that this model improves testing performance by 2% phoneme error rate absolute in low-resource conditions. Additionally, because we are explicitly modeling language-independent phones, we can build a (nearly-)universal phone recognizer that, when combined with the PHOIBLE [1] large, manually curated database of phone inventories, can be customized into 2,000 language dependent recognizers. Experiments on two low-resourced indigenous languages, Inuktitut and Tusom, show that our recognizer achieves phone accuracy improvements of more than 17%, moving a step closer to speech recognition for all languages in the world. 1

Acoustics Based Intent Recognition Using Discovered Phonetic Units for Low Resource Languages

“…Our work is closest to [9] where authors discover symbolic units in an unsupervised fashion for speech to speech translation. Contrary to this work, we employ the symbolic units generated by Allosaurus [10] which is trained in a supervised fashion.…”

“…The dataset in each language contains two voices -one male and one female. The audios are then passes into Allosaurus [10] to discover phonetic units and create a phonetic transcription of the audio.…”

“…The input audio is directly fed into a system that can generate hypothesized phonetic units. For our work, we use the Allosaurus library [10] which is a nearly-universal phone recognition system. For this work, we employ the language dependent phones, which basically means we're providing an identifier to Allosaurus for audio language.…”

ALICE: An open-source tool for automatic measurement of phoneme, syllable, and word counts from child-centered daylong recordings

“…Similarly to LENA (Xu et al, 2008), we explore the use of automatic phone recognition as a feature extractor for ALUC estimation. For this purpose, we use Allosaurus 2 , a languageindependent phone recognizer (Li et al, 2020). Allosaurus is a multilayer Long Short-Term Memory (LSTM) neural network model trained on 12 different languages for phone and phoneme recognition.…”

Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpus

“…For ASR it is possible to combine a target-language language model with an acoustic model from a phonologically similar language, with no need for parallel datasets of audio recordings and transcriptions . Such approaches are likely to get even more effective with nearly-universal acoustic models (Li et al, 2020) and more scalable grapheme-to-phoneme modeling approaches (Deri and Knight, 2016;Mortensen et al, 2018;Bleyan et al, 2019;Ritchie et al, 2020;. Even if more work is needed to establish when such approaches will work well (Marchisio et al, 2020;Artetxe et al, 2020;Wu and Dredze, 2020), having useful monolingual text corpora across languages is clearly a prerequisite to exploring such approaches further.…”