An algorithm for maximum likelihood estimation of hidden Markov models with unknown state-tying

Abstract

For speech recognition based on hidden Markov modeling, parameter-tying, which consists in constraining some of the parameters of the model to share the same value, has emerged as a standard practice. In this paper, an original algorithm is proposed that makes it possible to jointly estimate both the shared model parameters and the tying characteristics, using the maximum likelihood criterion. The proposed algorithm is based on a recently introduced extension of the classic expectationmaximization (EM) framework. The convergence properties of this class of algorithms are analyzed in detail. The method is evaluated on an isolated word recognition task using hidden Markov models (HMM's) with Gaussian observation densities and tying at the state level. Finally, the extension of this method to the case of mixture observation densities with tying at the mixture component level is discussed.

Substate tying with combined parameter training and reduction in tied-mixture HMM design

“…The marginal probability of a universal codebook element is (11) where (12) Consider the posterior probability (13) where the last approximation is valid if the states are roughly equiprobable. Note that in general the criterion will be calculated without the approximation, but complexity can be saved when it is valid.…”

“…This extreme measure yields substantial complexity reduction, but may cause serious degradation in model accuracy. Second, although efficient optimization algorithms have been proposed for state tying [10]- [13], they are typically initialized in a greedy suboptimal fashion. This may impact the performance, especially in the case of a large number of Markov states.…”

Resilient Dynamic Power Management under Uncertainty

“…The EM algorithm is a general technique which can be used to determine the maximum likelihood estimate (MLE) of the parameters of an underlying distribution from some given data when the observed data is incomplete [19]. Alternatively stated, it is used to find the MLE of some parameters in a probabilistic model, where the model depends on unobserved latent (hidden) variables.…”

Speech enhancement by ARUI model and a new type of adaptive filter with sequential processing

“…Speech enhancement should be considered when an application requires better quality and higher performance [1][2][3][4][5][6][7][8]. The speech enhancement problem consists of a family of problems characterized by the type of noise source, the way in which the noise interacts with the clean signal, the number of voice channels or microphone outputs available for enhancement and the nature of the speech communication system.…”

“…In the last two decades, there have been three major classes of speech enhancement algorithm based on 1) hidden Markov modeling (HMM) of noise and/or speech signal [1,2], 2) transformation of signals such as spectral subtraction [3][4][5], and 3) time domain filtering such as adaptive noise reduction.…”