Polyphone disambiguation and accent prediction using pre-trained language models in Japanese TTS front-end

Abstract

Although end-to-end text-to-speech (TTS) models can generate natural speech, challenges still remain when it comes to estimating sentence-level phonetic and prosodic information from raw text in Japanese TTS systems. In this paper, we propose a method for polyphone disambiguation (PD) and accent prediction (AP). The proposed method incorporates explicit features extracted from morphological analysis and implicit features extracted from pre-trained language models (PLMs). We use BERT and Flair embeddings as implicit features and examine how to combine them with explicit features. Our objective evaluation results showed that the proposed method improved the accuracy by 5.7 points in PD and 6.0 points in AP. Moreover, the perceptual listening test results confirmed that a TTS system employing our proposed model as a front-end achieved a mean opinion score close to that of synthesized speech with groundtruth pronunciation and accent in terms of naturalness.

Dict-TTS: Learning to Pronounce with Prior Dictionary Knowledge for Text-to-Speech

“…The pronunciation of a polyphone is defined by the semantic context information of neighbouring characters [46]. In order to comprehend the semantic meaning in the given sentence for polyphone disambiguation, previous methods [11,53,45,18,8] have adopted the pre-trained language model [12] to extract semantic features from raw character sequences and predict the pronunciation of polyphones with neural classifiers according to the semantic features. Among them, PnG BERT and Mixed-Phoneme BERT [22,58] take both phoneme and grapheme as input to train an augmented BERT and use the pre-trained augmented BERT as the TTS encoder.…”

“…Capturing the pronunciations from raw texts is challenging for end-to-end text-to-speech (TTS) systems [2,28,38,41,51,24,14,37], since there are full of words that are not covered by general pronunciation rules [4,21,46]. Therefore, polyphone 3 disambiguation (one of the biggest challenges in converting texts into phonemes [33,56,42]) plays an important role in the construction of highquality neural TTS systems [18,34]. However, since the exact pronunciation of a polyphone must be The illustration of the dictionary entry that contains information on character's or word's definitions, usages, and pronunciations.…”