HMM topology design using maximum likelihood successive state splitting

Ostendorf, Mari; Singer, Harald

doi:10.1006/csla.1996.0021

Cited by 103 publications

(68 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In separate work, Sagayama et al [31] have proposed asynchronous transition HMMs (AT-HMMs) which model the temporal characteristics of each acoustic feature component separately. Their system uses a form of the successive state splitting algorithm [34,26] to learn the temporal and contextual characteristics of each feature. Using Mel-scale cepstra as observations, they report a significant reduction of errors compared to a standard HMM approach.…”

Section: Discussionmentioning

confidence: 99%

Detection of phonological features in continuous speech using neural networks

King

Taylor

2000

Computer Speech & Language

158

124

View full text Add to dashboard Cite

We report work on the first component of a two stage speech recognition architecture based on phonological features rather than phones. The paper reports experiments on three phonological feature systems: 1) the Sound Pattern of English (SPE) system which uses binary features, 2) a multi valued (MV) feature system which uses traditional phonetic categories such as manner, place etc, and 3) Government Phonology (GP) which uses a set of structured primes. All experiments used recurrent neural networks to perform feature detection. In these networks the input layer is a standard framewise cepstral representation, and the output layer represents the values of the features. The system effectively produces a representation of the most likely phonological features for each input frame.All experiments were carried out on the TIMIT speaker independent database. The networks performed well in all cases, with the average accuracy for a single feature ranging from 86% and 93%. We describe these experiments in detail, and discuss the justification and potential advantages of using phonological features rather than phones for the basis of speech recognition.

show abstract

Section: Discussionmentioning

confidence: 99%

Detection of phonological features in continuous speech using neural networks

King

Taylor

2000

Computer Speech & Language

158

124

View full text Add to dashboard Cite

show abstract

“…Ikeda presented a similar scheme with an objective function built around Aikake's Information Criterion to limit over-fitting [Ikeda, 1993]. The speech recognition literature now contains numerous variants of this strategy, including maximum likelihood criteria for splitting [Ostendorf and Singer, 1997]; search by genetic algorithms [Yada et al, 1996, Takara et al, 1997; and splitting to describe exceptions [Fujiwara et al, 1995, Valtchev et al, 1997. Nearly all of these algorithms use beam search (generate-andtest with multiple heads) to compensate for dead-ends and declines in posterior probability; most of the computation is squandered.…”

Section: Hmm Inductionmentioning

confidence: 99%

Structure Learning in Conditional Probability Models via an Entropic Prior and Parameter Extinction

Brand

1999

Neural Computation

126

View full text Add to dashboard Cite

We introduce an entropic prior for multinomial parameter estimation problems and solve for its maximum a posteriori (MAP) estimator. The prior is a bias for maximally structured and minimally ambiguous models. In conditional probability models with hidden state, iterative MAP estimation drives weakly supported parameters toward extinction, effectively turning them off. Thus structure discovery is folded into parameter estimation. We then establish criteria for simplifying a probabilistic model's graphical structure by trimming parameters and states, with a guarantee that any such deletion will increase the posterior probability of the model. Trimming accelerates learning by sparsifying the model. All operations monotonically and maximally increase the posterior probability, yielding structure-learning algorithms only slightly slower than parameter estimation via expectation-maximization (EM), and orders of magnitude faster than search-based structure induction. When applied to hidden Markov model (HMM) training, the resulting models show superior generalization to held-out test data. In many cases the resulting models are so sparse and concise that they are interpretable, with hidden states that strongly correlate with meaningful categories. Neural ComputationThis work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. AbstractWe introduce an entropic prior for multinomial parameter estimation problems and solve for its maximum a posteriori (MAP) estimator. The prior is a bias for maximally structured and minimally ambiguous models. In conditional probability models with hidden state, iterative MAP estimation drives weakly supported parameters toward extinction, effectively turning them off. Thus structure discovery is folded into parameter estimation. We then establish criteria for simplifying a probabilistic model's graphical structure by trimming parameters and states, with a guarantee that any such deletion will increase the posterior probability of the model. Trimming accelerates learning by sparsifying the model. All operations monotonically and maximally increase the posterior probability, yielding structure-learning algorithms only slightly slower than parameter estimation via expectation-maximization (EM), and orders of magnitude faster than search-based structure induction. When applied to hidden Markov model (HMM) training, the resulting models show superior generalization to held-ou...

show abstract

“…Furthermore, it is impossible for a finite set of training data to cover every possible combination of contextual factors. Various parameter-tying techniques have been developed [7]- [11] to avoid this problem. Among them, a decision tree-based context-clustering technique [12] has been widely used.…”

Section: Introductionmentioning

confidence: 99%

A Covariance-Tying Technique for HMM-Based Speech Synthesis

Oura

Zen

Nankaku

et al. 2010

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYA technique for reducing the footprints of HMM-based speech synthesis systems by tying all covariance matrices of state distributions is described. HMM-based speech synthesis systems usually leave smaller footprints than unit-selection synthesis systems because they store statistics rather than speech waveforms. However, further reduction is essential to put them on embedded devices, which have limited memory. In accordance with the empirical knowledge that covariance matrices have a smaller impact on the quality of synthesized speech than mean vectors, we propose a technique for clustering mean vectors while tying all covariance matrices. Subjective listening test results showed that the proposed technique can shrink the footprints of an HMM-based speech synthesis system while retaining the quality of the synthesized speech.

show abstract

HMM topology design using maximum likelihood successive state splitting

Cited by 103 publications

References 21 publications

Detection of phonological features in continuous speech using neural networks

Detection of phonological features in continuous speech using neural networks

Structure Learning in Conditional Probability Models via an Entropic Prior and Parameter Extinction

A Covariance-Tying Technique for HMM-Based Speech Synthesis

Contact Info

Product

Resources

About