Acoustic data-driven grapheme-to-phoneme conversion in the probabilistic lexical modeling framework

Abstract

One of the primary steps in building automatic speech recognition (ASR) as well as text-to-speech systems is development of a phonemic lexicon that provides a mapping between each word and its pronunciation as a sequence of phonemes. Phoneme lexicons can be developed by humans through use of linguistic knowledge, however, this would be a costly and time-consuming task. To facilitate this process, grapheme-to-phoneme conversion (G2P) techniques are used in which given an initial phoneme lexicon, the relationship between graphemes and phonemes is learned through data-driven methods. This article presents a novel G2P formalism which learns the grapheme-to-phoneme relationship through acoustic data and potentially relaxes the need for an initial phonemic lexicon in the target language of interest. The formalism involves a training part followed by an inference part. In the training part, the graphemeto-phoneme relationship is captured in a probabilistic lexical modeling framework. Within this framework, a hidden Markov model (HMM) is trained in which each HMM state representing a grapheme is parameterized by a categorical distribution of phonemes. Then in the inference part, given the orthographic transcription of the word and the learned HMM, the most probable sequence of phonemes is inferred. In this article, we show that the recently proposed acoustic G2P approach in the Kullback-Leibler divergence-based HMM (KL-HMM) framework is a particular case of this formalism. We then benchmark the approach against two popular G2P approaches, namely joint multigram approach and decision tree-based approach. Our experimental studies on English and French show that despite relatively poor performance at the pronunciation level, the proposed approach can achieve comparable performance to state-ofthe-art G2P methods at the ASR level.

Testing the Robustness of Laws of Polysemy and Brevity Versus Frequency

“…An accurate measurement of brevity would require detailed acoustical information that is missing in raw written transcripts [10] or using more sophisticated methods of computation, for instance, to calculate number of phonemes and syllables according to [1]. However, the relationship between the duration of phonemes and graphemes is well-known and in general longer words has longer durations: grapheme-to-phoneme conversion is still a hot topic of research, due to the ambiguity of graphemes with respect to their pronunciation that today supposes a difficulty in speech technologies [18]. In order to improve the frequency measure, we would consider the use of alternative databases, e.g., the frequency of English words in Wikipedia [11].…”

A Posterior-Based Multistream Formulation for G2P Conversion

“…In both cases though the performance is statistically comparable. This trend is more attributed to the fact that acoustic G2P conversion approach typically leads to acoustically confusable substitutions [20], which a discriminative acoustic model (ANN) seems to handle better than a generative acoustic model (GMM). Finally, the best performance of 93.1%…”

“…In case of DTs, the estimates are Kronecker delta distributions [20], as DTs map a central grapheme with contextual information deterministically onto a phoneme.…”

“…The acoustic feature was 39 dimensional PLP cepstral features (c 0 − c 12 + ∆ + ∆∆) extracted using HTK [32]. Following the observations made in [20], we used G2P generated lexicons to train the ASR system, as it yields better systems than the case when trained with manual lexicon and tested with G2P lexicon. The number of tied states were between 2174 and 2270.…”

“…Towards that, we first present a posterior based G2P conversion formalism analogous to hybrid HMM/ANN ASR approach, originally proposed in [20] (Section II). We then show how multiple estimates of P (f |g), probability of phoneme f given grapheme g, can be estimated through different techniques and combined in a multi-stream fashion, exactly as done in ASR, for G2P conversion.…”