Requirements and Motivations of Low-Resource Speech Synthesis for Language Revitalization

Abstract

This paper describes the motivation and development of speech synthesis systems for the purposes of language revitalization. By building speech synthesis systems for three Indigenous languages spoken in Canada, Kanien'kéha, Gitksan & SENĆOŦEN, we re-evaluate the question of how much data is required to build low-resource speech synthesis systems featuring state-of-the-art neural models. For example, preliminary results with English data show that a FastSpeech2 model trained with 1 hour of training data can produce speech with comparable naturalness to a Tacotron2 model trained with 10 hours of data. Finally, we motivate future research in evaluation and classroom integration in the field of speech synthesis for language revitalization.

A Low-Resource Pipeline for Text-to-Speech from Found Data With Application to Scottish Gaelic

“…We train 7 different FastPitch [12] acoustic models using either 2 or 8 hours of speech to predict mel-scaled spectrogram features from character, phone or discrete acoustic unit sequences, as described in the following sections. Following [21,1], we replace all convolutional layers with depthwise-separable convolutions, reducing overall parameter counts to match our lowdata setting. Character-and phone-input models are trained with target durations from forced alignments, while for acoustic unit sequences we derive target durations by run-length encoding repeated consecutive units.…”

“…Data requirements for neural TTS have typically been put at some tens of hours of studio-quality speech recordings paired with text transcripts, although recent work has reevaluated these assumptions by switching to non-autoregressive architectures where the burden of learning text-speech alignments alongside acoustic feature prediction is removed [1], or by using powerful self-supervised speech representations to help train some parts of the system on noisier audio data [2]. Much effort has also been directed toward using 'found' data not originally intended for TTS, especially audiobook recordings.…”

“…In this work we develop a replicable and open-source recipe for building a speech corpus and neural TTS system for Scottish Gaelic, all based on publicly-available data. 1 Our work is similar in motivation and method to [3], applied in a more recent TTS framework. We begin by segmenting and aligning over 85 hours of short speech utterances and corresponding text transcripts from long-form audio recordings found online, using a purely character-based acoustic model.…”

A Systematic Review and Analysis of Multilingual Data Strategies in Text-to-Speech for Low-Resource Languages

“…Although such finetuning is obviously still advantageous, another more likely scenario for LRLs is that researchers could find a few enthusiastic listeners that would not mind rating many samples, rather than the other way around. Besides, the work by [26] shows that, for LRLs with very small communities, the evaluation of a few community-engaged and respected speakers of the language can be representative of that of the whole community.…”