Neural Machine Translation for Multilingual Grapheme-to-Phoneme Conversion

Sokolov, Alex; Rohlin, Tracy; Rastrow, Ariya

doi:10.21437/interspeech.2019-3176

Cited by 9 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Tan et al [14] proposed to train separate models for each language at first and then perform knowledge distillation from each language-specific model to the multilingual model for multilingual translation. Wang et al [12] presented a Grapheme-to-Phoneme (G2P) model which share the same encoder and decoder across multiple languages by utilizing a combination of universal symbol inventories of Latin-like alphabets and cross-linguistically shared feature representations. Toshniwal et al [15] take a union of language-specific grapheme sets and train a grapheme-based sequence-to-sequence model on data combined by different languages for speech recognition.…”

Section: Multilingual Learningmentioning

confidence: 99%

Synchronous Bidirectional Learning for Multilingual Lip Reading

Luo,

Yang,

Chen

et al. 2020

Preprint

View full text Add to dashboard Cite

Lip reading has received increasing attention in recent years. This paper focuses on the synergy of multilingual lip reading. There are more than 7,000 languages in the world, which implies that it is impractical to train separate lip reading models by collecting large-scale data per language. Although each language has its own linguistic and pronunciation features, the lip movements of all languages share similar patterns. Based on this idea, in this paper, we try to explore the synergized learning of multilingual lip reading, and further propose a synchronous bidirectional learning (SBL) framework for effective synergy of multilingual lip reading. Firstly, we introduce the phonemes as our modeling units for the multilingual setting. Similar phoneme always leads to similar visual patterns. The multilingual setting would increase both the quantity and the diversity of each phoneme shared among different languages. So the learning for the multilingual target should bring improvement to the prediction of phonemes. Then, a SBL block is proposed to infer the target unit when given its previous and later context. The rules for each specific language which the model itself judges to be is learned in this fill-in-the-blank manner. To make the learning process more targeted at each particular language, we introduce an extra task of predicting the language identity in the learning process. Finally, we perform a thorough comparison on LRW (English) and LRW-1000 (Mandarin).. The results outperform the existing state of the art by a large margin, and show the promising benefits from the synergized learning of different languages.

show abstract

Section: Multilingual Learningmentioning

confidence: 99%

Synchronous Bidirectional Learning for Multilingual Lip Reading

Luo,

Yang,

Chen

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Furthermore, the efficacy of end-to-end automatic speech recognition (ASR) models utilizing character or word-based modeling units is limited in terms of effectively accommodating out-of-vocabulary (OOV), unless pronunciation dictionaries are employed. As a result, an increasing number of scholars are exploring the development of multilingual pronunciation dictionaries [1][2][3][4][5]. A multilingual pronunciation dictionary is constructed to solve the pronunciation problem in multilingual speech technology.…”

Section: Introductionmentioning

confidence: 99%

“…For instance, when a low-resource language shares certain phonemes with a high-resource language, an existing pronunciation dictionary for the latter can assist in constructing a pronunciation dictionary for the former. Primarily, this methodology has been proven to be advantageous for low-resource languages constrained by limited data [5,6].…”

Section: Introductionmentioning

confidence: 99%

Near-Optimal Active Learning for Multilingual Grapheme-to-Phoneme Conversion

Cao,

Zhao,

2023

Applied Sciences

View full text Add to dashboard Cite

The construction of pronunciation dictionaries relies on high-quality and extensive training data in data-driven way. However, the manual annotation of corpus for this purpose is both costly and time consuming, especially for low-resource languages that lack sufficient data and resources. A multilingual pronunciation dictionary includes some common phonemes or phonetic units, which means that these phonemes or units have similarities in the pronunciation of different languages and can be used in the construction process of pronunciation dictionaries for low-resource languages. By using a multilingual pronunciation dictionary, knowledge can be shared among different languages, thus improving the quality and accuracy of pronunciation dictionaries for low-resource languages. In this paper, we propose using shared articulatory features among multiple languages to construct a universal phoneme set, which is then used to label words for multiple languages. To achieve this, we first developed a grapheme−phoneme (G2P) model based on an encoder−decoder deep neural network. We then adopted a near-optimal active learning method in the process of building the pronunciation dictionary to select informative samples from a large, unlabeled corpus and had them labeled by experts. Our experiments demonstrate that this method selected about 1/5 of the unlabeled data and achieved an even higher conversion accuracy than the results of the large data training method. By selectively labeling samples with a high uncertainty in the model, while avoiding labeling samples that were accurately predicted by the current model, our method greatly enhances the efficiency of pronunciation dictionary construction.

show abstract

Literature Review of Research on Common Methods of Grapheme-To-Phoneme

Zhang,

Cao

2023

Lecture Notes in Operations Research

View full text Add to dashboard Cite

Neural Machine Translation for Multilingual Grapheme-to-Phoneme Conversion

Cited by 9 publications

References 15 publications

Synchronous Bidirectional Learning for Multilingual Lip Reading

Synchronous Bidirectional Learning for Multilingual Lip Reading

Near-Optimal Active Learning for Multilingual Grapheme-to-Phoneme Conversion

Literature Review of Research on Common Methods of Grapheme-To-Phoneme

Contact Info

Product

Resources

About