Language-Agnostic Meta-Learning for Low-Resource Text-to-Speech with Articulatory Features

Lux, Florian; Vu, Thang X.

doi:10.18653/v1/2022.acl-long.472

Cited by 14 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To accommodate a wider range of languages, recent studies have also focused on low-resource languages. Collecting a sufficient amount of high-quality paired training data for lowresource languages can be challenging; therefore, previous work has adapted a TTS model trained on resource-rich languages, to low-resource languages [5], [6], [7], [21]. This approach first pretrains a multilingual TTS model and then fine-tune it for low-resource languages, thereby improving performance by exploiting the multilingual knowledge embedded in the pretrained TTS model.…”

Section: A Low-resource Language Adaptation For Ttsmentioning

confidence: 99%

“…While this method requires an ASR model trained on speech corpora of the target language, our approach adapts language-aware embedding layer only using text data with MLM objectives. Previous multilingual TTS work has focused on tokens, including bytes [24], [7], [9], [11], IPA symbols [25], [26], [11], articulatory features [21], sentencepiece tokens [9], etc. Our work proposes a graphone-based training method for multilingual low-resource TTS, which allows the flexible use of both graphemes and IPA symbols for each language.…”

Section: B Other Multilingual Low-resource Tts Approachesmentioning

confidence: 99%

See 1 more Smart Citation

Text-Inductive Graphone-Based Language Adaptation for Low-Resource Speech Synthesis

Saeki,

Maiti,

et al. 2024

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Neural text-to-speech (TTS) systems have made significant progress in generating natural synthetic speech. However, neural TTS requires large amounts of paired training data, which limits its applicability to a small number of resource-rich languages. Previous work on low-resource TTS has addressed the data hungriness based on transfer learning from a multilingual model to low-resource languages, but it still relies heavily on the availability of paired data for the target languages. In this paper, we propose a text-inductive language adaptation framework for low-resource TTS to address the cost of collecting the paired data for low-resource languages. To inject textual knowledge during transfer learning, our framework employs a two-stage adaptation scheme that utilizes both text-only and supervised data for the target language. In the text-based adaptation stage, we update the language-aware embedding layer with a masked language model objective using text-only data for the target language. In the supervised adaptation stage, the entire TTS model is updated using paired data for the target language. We also propose a graphone-based multilingual training method that jointly uses graphemes and International Phonetic Alphabet symbols (referred to as graphones) for resource-rich languages, while using only graphemes for low-resource languages. This approach facilitates the transfer of pronunciation knowledge from resource-rich to low-resource languages. Through extensive evaluations, we demonstrate that 1) our framework with textbased adaptation outperforms the previous supervised transfer learning approach, 2) the proposed graphone-based training method further improves the performance of both multilingual TTS and low-resource language adaptation. With only 5 minutes of paired data for fine-tuning, our method achieved highly intelligible synthetic speech with the character error rates of around 6 % for a target language.

show abstract

Section: A Low-resource Language Adaptation For Ttsmentioning

confidence: 99%

Section: B Other Multilingual Low-resource Tts Approachesmentioning

confidence: 99%

Text-Inductive Graphone-Based Language Adaptation for Low-Resource Speech Synthesis

Saeki,

Maiti,

et al. 2024

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…The relative relationships are more reliable in the AMOS. Vu, 2022], to transfer knowledge from resource-rich to lowresource languages. Grapheme tokens can eliminate the perlanguage G2P knowledge, and previous work has built a bytebased TTS model for around 40 languages .…”

Section: Related Workmentioning

confidence: 99%

Learning to Speak from Text: Zero-Shot Multilingual Text-to-Speech with Unsupervised Text Pretraining

Saeki¹,

Maiti²,

Li³

et al. 2023

Preprint

View full text Add to dashboard Cite

While neural text-to-speech (TTS) has achieved human-like natural synthetic speech, multilingual TTS systems are limited to resource-rich languages due to the need for paired text and studio-quality audio data. This paper proposes a method for zeroshot multilingual TTS using text-only data for the target language. The use of text-only data allows the development of TTS systems for low-resource languages for which only textual resources are available, making TTS accessible to thousands of languages. Inspired by the strong cross-lingual transferability of multilingual language models, our framework first performs masked language model pretraining with multilingual text-only data. Then we train this model with a paired data in a supervised manner, while freezing a language-aware embedding layer. This allows inference even for languages not included in the paired data but present in the text-only data. Evaluation results demonstrate highly intelligible zero-shot TTS with a character error rate of less than 12% for an unseen language. All experiments were conducted using public datasets and the implementation will be made available for reproducibility.

show abstract

“…Recently, optimization-based techniques yield substantial improvement in many low-resource NLP tasks (Zhao et al, 2022). Among them, Model-Agnostic Meta-Learning (MAML) (Finn et al, 2017) has been widely used to tackle low-resource NLP tasks such as machine translation (Gu et al, 2018;Park et al, 2021), dialogue generation (Mi et al, 2019;, and text-to-speech alignment (Lux and Vu, 2022). MAML has shown exceptional efficacy in learning a good parameter initialization for a fast adaption with limited resources.…”

Section: Related Workmentioning

confidence: 99%

Towards Low-Resource Automatic Program Repair with Meta-Learning and Pretrained Language Models

Wang,

Hoi

et al. 2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Automatic program repair (APR) has gained increasing attention as an essential technique in software development to reduce manual debugging efforts and boost developers' productivity. Recent advances in deep learning (DL) based models have demonstrated promising results by learning from large-scale bug-fix examples in a data-driven manner. However, in practical scenarios, software bugs have an imbalanced distribution, and the fixing knowledge learned by APR models often only capture the patterns of frequent error types, making it inapplicable to handle the rare error types. To address this limitation, we investigate a novel task of low-resource APR, and propose Meta-APR, a new meta-learning framework integrated with code pretrained language models to generate fixes for low-resource bugs with limited training samples. Our Meta-APR learns better errorspecific knowledge from high-resource bugs through efficient first-order meta-learning optimization, which allows for a faster adaptation to the target low-resource bugs. Besides, while we adopt CodeT5, a pretrained code-aware encoder-decoder Transformer, as the backbone model for Meta-APR, it is a model-agnostic framework that can be integrated with any neural models. Extensive experimental results on three benchmarks in various programming languages verify the superiority of our method over existing DL-based APR approaches.

show abstract

Language-Agnostic Meta-Learning for Low-Resource Text-to-Speech with Articulatory Features

Cited by 14 publications

References 0 publications

Text-Inductive Graphone-Based Language Adaptation for Low-Resource Speech Synthesis

Text-Inductive Graphone-Based Language Adaptation for Low-Resource Speech Synthesis

Learning to Speak from Text: Zero-Shot Multilingual Text-to-Speech with Unsupervised Text Pretraining

Towards Low-Resource Automatic Program Repair with Meta-Learning and Pretrained Language Models

Contact Info

Product

Resources

About