A Comparison of Letters and Phones as Input to Sequence-to-Sequence Models for Speech Synthesis

Abstract

Neural sequence-to-sequence (S2S) models for text-tospeech synthesis (TTS) may take letter or phone input sequences. Since for many languages phones have a more direct relationship to the acoustic signal, they lead to improved quality. But generating phone transcriptions from text requires an expensive dictionary and an error-prone grapheme-to-phoneme (G2P) model, and the relative improvement over using letters has yet to be quantified. In approaching this question, we presume that letter-input S2S models must implicitly learn an internal counterpart to G2P conversion and therefore inevitably make errors. Such a model may thus be viewed as phone-input S2S with inaccurate phone input. To quantify this inaccuracy, we compare in this paper a letter-input S2S system to several phone-input systems trained on data with a varying level of error in the phonetic transcription. Our findings show our letterinput system is equivalent in quality to the phone-input system in which 25% of word tokens in the training data have incorrect phonetic transcriptions. Furthermore, we find that for phoneinput systems up to 15% of word tokens in the training data can have incorrect phonetic transcriptions without any significant difference in performance to a 0% error rate system. This suggests it is acceptable to use G2P to predict pronunciations for out-of-vocabulary words (OOVs) provided they are less than around 15% of the training data, removing the need to manually add OOVs to the dictionary for every new training set.

Enhancing Sequence-to-Sequence Text-to-Speech with Morphology

“…System P system outperformed system G, repeating our previous findings from [10] with a different Tacotron implementation and vocoder. This is because phones map pronunciations directly unlike letters, leading to fewer ambiguities at training and test time.…”

“…In previous work [10], we found phone input significantly improved the quality of Ophelia 3 a different S2S architecture, based on DCTTS [26]. Here, we also measure the effect of morphemes when using phone-based input.…”

“…Mulitple reports have in fact been emerging in the literature showing that S2S TTS incorporating linguistic features can give superior performance. Indeed, state-of-the-art quality has only so far been achieved for English with phonetic input [10]. Syntactic features improved intonation in [11].…”

Cross-lingual Transfer of Phonological Features for Low-resource Speech Synthesis

“…Phonemes are often used as atomic input symbols to text-tospeech (TTS) systems as an explicit representation of the pronunciation of input text [1]. This is useful even for large neural sequence-to-sequence models which have the capacity to learn implicit pronunciation models directly from text inputs but which may make mistakes compared to grapheme-to-phoneme (g2p) conversion models trained on high-quality lexicons [2,3]. Such large TTS models are typically trained using tens of hours of audio data with associated text transcriptions, which alongside the specialist linguistic knowledge required to convert raw text into phoneme strings are expensive resources to attain and limit the application of these models to a small proportion of the world's 7,000 languages.…”

Cross-lingual Low Resource Speaker Adaptation Using Phonological Features

“…Recently proposed neural architectures [1,2] have shown that an efficient end-to-end acoustic model is possible by directly consuming text characters. The inputs to state-of-the-art TTS systems consist of either text characters (graphemes) or phonemes, with the superiority of phoneme-based systems recently quantified [3]. In multilingual TTS, these inputs may originate from various speakers and languages introducing variable factors in the model's logic.…”