Fast and flexible Kullback-Leibler divergence based acoustic modeling for non-native speech recognition

Rasipuram, Ramya; Magimai.-Doss, Mathew

doi:10.1109/asru.2011.6163956

Cited by 19 publications

(40 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 1 provides an overview of the systems that are investigated. The non-native and minority language ASR studies build on top of our preliminary investigations that focussed on KL-HMM and the use of word-internal contextdependent subword units (Imseng et al, 2011;.…”

Section: Methodsmentioning

confidence: 99%

“…In the light of that, previous works on KL-HMM such as (Imseng et al, 2011(Imseng et al, , 2012 suggest that ASR systems can be rapidly developed using language-independent acoustic model and training only the lexical model on target language or domain data.…”

Section: Potential Of Probabilistic Lexical Modelingmentioning

confidence: 99%

“…For instance, as in previous work on KL-HMM Imseng et al, 2011Imseng et al, , 2012, L can be based on context-dependent subword units while A can be based on context-independent subword units. These ASR systems have been found to yield performance comparable to or better than standard context-dependent subword unit based HMM/GMM system.…”

Section: Potential Of Probabilistic Lexical Modelingmentioning

confidence: 99%

“…The lexical model can capture the relationship between the distinct subword unit sets through acoustics. This flexibility has been exploited to build ASR systems where the acoustic unit set A is based on phones and the acoustic model is trained on auxiliary acoustic and lexical resources, and the lexical unit set L is based on graphemes and the lexical model is learned on target language or domain data Imseng et al, 2011;. It has been observed that this grapheme-based ASR approach could result in better or comparable performance even for languages such as English (where grapheme-to-phoneme relationship is irregular) compared to two stage approach where G2P training is followed by ASR system development ).…”

Section: Potential Of Probabilistic Lexical Modelingmentioning

confidence: 99%

“…In a recent work, we showed that there are probabilistic lexical modeling approaches such as, Kullback-Leibler divergence based hidden Markov model (KL-HMM) where the relationship between lexical units and acoustic units is probabilistic . Probabilistic lexical modeling relaxes certain constraints imposed by deterministic lexical modeling and as a consequence, acoustic model and lexical model can be independently trained on different set of resources (Imseng et al, 2011(Imseng et al, , 2012; different kinds of subword units can be modeled in an ASR system Imseng et al, 2011; and different types of contextual units can be modeled in an ASR system Imseng et al, 2011Imseng et al, , 2012. Motivated by these findings, this paper proposes an approach for rapid development of ASR systems in the framework of probabilistic lexical modeling with minimal acoustic and lexical resources from target language or domain.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Acoustic and lexical resource constrained ASR using language-independent acoustic model and language-dependent probabilistic lexical model

Rasipuram

Magimai-Doss

2015

Speech Communication

Self Cite

View full text Add to dashboard Cite

One of the key challenge involved in building a statistical automatic speech recognition (ASR) system is modeling the relationship between lexical units (that are based on subword units in the pronunciation lexicon) and acoustic feature observations. To model this relationship two types of resources are needed, namely, acoustic resources (speech signals with word level transcriptions) and lexical resources (which transcribes each word in terms of subword units). Standard ASR systems typically use phonemes or phones as subword units. Not all languages have well developed acoustic resources and phonetic lexical resources. In this paper, we show that modeling of the relationship between lexical units and acoustic features can be factored into two parts through a latent variable, referred to as acoustic units, namely: (a) acoustic model that models the relationship between acoustic features and acoustic units and (b) lexical model that models the relationship between lexical units and acoustic units. Through this understanding, we elucidate that in standard hidden Markov model (HMM) based ASR system, the lexical model is deterministic (i.e., there exists an one-to-one relationship between lexical units and acoustic units), and it is the deterministic lexical model that imposes the need for well developed acoustic and lexical resources in the target language or domain when building ASR system. We then propose an approach that addresses both acoustic resource and lexical resource constraints. More specifically, in the proposed approach the acoustic model models the relationship between acoustic features and multilingual phones (acoustic units) on target language-independent data, and the lexical model models a probabilistic relationship between lexical units based on graphemes and multilingual phones on small amount of target language data. We show the potential and the efficacy of the proposed approach through experiments and comparisons with other approaches on three different ASR tasks, namely, non-native accented speech recognition, rapid development of ASR system for a new language and development of ASR system for a minority language.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Potential Of Probabilistic Lexical Modelingmentioning

confidence: 99%

Section: Potential Of Probabilistic Lexical Modelingmentioning

confidence: 99%

Section: Potential Of Probabilistic Lexical Modelingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Acoustic and lexical resource constrained ASR using language-independent acoustic model and language-dependent probabilistic lexical model

Rasipuram

Magimai-Doss

2015

Speech Communication

Self Cite

View full text Add to dashboard Cite

show abstract

Automatic Speech Recognition

Renals¹,

King²

2010

The Handbook of Phonetic Sciences

View full text Add to dashboard Cite

In most of state-of-the-art speech recognition systems, Gaussian mixture models (GMMs) are used to model the density of the emitting states in the hidden Markov models (HMMs). In a conventional system, the model parameters of each GMM are estimated directly and independently given the alignment. This results a large number of model parameters to be estimated, and consequently, a large amount of training data is required to fit the model. In addition, different sources of acoustic variability that impact the accuracy of a recogniser such as pronunciation variation, accent, speaker factor and environmental noise are only weakly modelled and factorized by adaptation techniques such as maximum likelihood linear regression (MLLR), maximum a posteriori adaptation (MAP) and vocal tract length normalisation (VTLN). In this thesis, we will discuss an alternative acoustic modelling approach -the subspace Gaussian mixture model (SGMM), which is expected to deal with these two issues better. In an SGMM, the model parameters are derived from low-dimensional model and speaker subspaces that can capture phonetic and speaker correlations. Given these subspaces, only a small number of state-dependent parameters are required to derive the corresponding GMMs. Hence, the total number of model parameters can be reduced, which allows acoustic modelling with a limited amount of training data. In addition, the SGMM-based acoustic model factorizes the phonetic and speaker factors and within this framework, other source of acoustic variability may also be explored.In this thesis, we propose a regularised model estimation for SGMMs, which avoids overtraining in case that the training data is sparse. We will also take advantage of the structure of SGMMs to explore cross-lingual acoustic modelling for low-resource speech recognition. Here, the model subspace is estimated from out-domain data and ported to the target language system. In this case, only the state-dependent parameters need to be estimated which relaxes the requirement of the amount of training data. To improve the robustness of SGMMs against environmental noise, we propose to apply the joint uncertainty decoding (JUD) technique that is shown to be efficient and effective. We will report experimental results on the Wall Street Journal (WSJ) database and GlobalPhone corpora to evaluate the regularisation and cross-lingual modelling of SGMMs. Noise compensation using JUD for SGMM acoustic models is evaluated on the Aurora 4 database.iii AcknowledgementsFirst and foremost, I would like to thank my supervisor Prof. Steve Renals for his expert guidance, deep insight in the speech technology, patience and encouragement all the way through my doctoral study. It is a great pleasure to work with this wonderful mentor. I also owe a huge gratitude to Dr. Arnab Ghoshal, a colleague, friend and advisor, without whom, I would definitely have a much tougher experience to reach this thesis. Thanks also to KK Chin -my industry supervisor in Toshiba Cambridge research laboratory (CRL). It is a...

show abstract

Acoustic data-driven grapheme-to-phoneme conversion in the probabilistic lexical modeling framework

Razavi

Rasipuram

Magimai.-Doss

2016

Speech Communication

Self Cite

View full text Add to dashboard Cite

One of the primary steps in building automatic speech recognition (ASR) as well as text-to-speech systems is development of a phonemic lexicon that provides a mapping between each word and its pronunciation as a sequence of phonemes. Phoneme lexicons can be developed by humans through use of linguistic knowledge, however, this would be a costly and time-consuming task. To facilitate this process, grapheme-to-phoneme conversion (G2P) techniques are used in which given an initial phoneme lexicon, the relationship between graphemes and phonemes is learned through data-driven methods. This article presents a novel G2P formalism which learns the grapheme-to-phoneme relationship through acoustic data and potentially relaxes the need for an initial phonemic lexicon in the target language of interest. The formalism involves a training part followed by an inference part. In the training part, the graphemeto-phoneme relationship is captured in a probabilistic lexical modeling framework. Within this framework, a hidden Markov model (HMM) is trained in which each HMM state representing a grapheme is parameterized by a categorical distribution of phonemes. Then in the inference part, given the orthographic transcription of the word and the learned HMM, the most probable sequence of phonemes is inferred. In this article, we show that the recently proposed acoustic G2P approach in the Kullback-Leibler divergence-based HMM (KL-HMM) framework is a particular case of this formalism. We then benchmark the approach against two popular G2P approaches, namely joint multigram approach and decision tree-based approach. Our experimental studies on English and French show that despite relatively poor performance at the pronunciation level, the proposed approach can achieve comparable performance to state-ofthe-art G2P methods at the ASR level.

show abstract

Fast and flexible Kullback-Leibler divergence based acoustic modeling for non-native speech recognition

Cited by 19 publications

References 11 publications

Acoustic and lexical resource constrained ASR using language-independent acoustic model and language-dependent probabilistic lexical model

Acoustic and lexical resource constrained ASR using language-independent acoustic model and language-dependent probabilistic lexical model

Automatic Speech Recognition

Acoustic data-driven grapheme-to-phoneme conversion in the probabilistic lexical modeling framework

Contact Info

Product

Resources

About