Learning mixture models using a genetic version of the EM algorithm

Martı́nez, Aleix M.; Virtriá, Jordi

doi:10.1016/s0167-8655(00)00031-3

Cited by 36 publications

(33 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Exceptions where both mean vectors and full covariance matrices were used include [4,5] where EM was used for the actual local optimization by fitting Gaussians to data in each iteration and GA was used only to guide the global search by selecting individual Gaussian components from existing candidate solutions in the reproduction steps. However, treating each Gaussian component as a whole in the search process and fitting it locally using the EM iterations may not explore the whole solution space effectively especially in higher dimensions.…”

Section: Related Workmentioning

confidence: 99%

“…These approaches enable multiple candidate solutions to simultaneously converge to possibly different optima by making use of the interactions. Genetic algorithm (GA) [3][4][5][6][7], differential evolution (DE) [8], and particle swarm optimization (PSO) [9][10][11][12] have been the most common population-based stochastic search algorithms used for the estimation of some form of GMMs. Even though these approaches have been shown to perform better than non-stochastic alternatives such as k-means and fuzzy c-means, the interaction mechanism that forms the basis of the power of the stochastic search algorithms has also limited the use of these methods due to some inherent assumptions in the candidate solution parametrization.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Maximum likelihood estimation of Gaussian mixture models using stochastic search

Ari

Aksoy

Arıkan

2012

Pattern Recognition

View full text Add to dashboard Cite

a b s t r a c tGaussian mixture models (GMM), commonly used in pattern recognition and machine learning, provide a flexible probabilistic model for the data. The conventional expectation-maximization (EM) algorithm for the maximum likelihood estimation of the parameters of GMMs is very sensitive to initialization and easily gets trapped in local maxima. Stochastic search algorithms have been popular alternatives for global optimization but their uses for GMM estimation have been limited to constrained models using identity or diagonal covariance matrices. Our major contributions in this paper are twofold. First, we present a novel parametrization for arbitrary covariance matrices that allow independent updating of individual parameters while retaining validity of the resultant matrices. Second, we propose an effective parameter matching technique to mitigate the issues related with the existence of multiple candidate solutions that are equivalent under permutations of the GMM components. Experiments on synthetic and real data sets show that the proposed framework has a robust performance and achieves significantly higher likelihood values than the EM algorithm.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Maximum likelihood estimation of Gaussian mixture models using stochastic search

Ari

Aksoy

Arıkan

2012

Pattern Recognition

View full text Add to dashboard Cite

show abstract

“…We use a kind of genetic EM (GA-EM) (Martínez and Virtriá 2000;Pernkopf and Bouchaffra 2005) combining EM with a Genetic Algorithm (GA) for estimating the models. A GA works by simulating evolution in natural systems, by initialization of a population of individuals and then iteratively performing selection of the fittest individuals and reproduction until a termination criterion is met.…”

Section: Mixture Model Estimationmentioning

confidence: 99%

“…GA-EM estimation therefore attempts to fix the convergence problem present in EM estimation. In contrast to earlier GA-EM algorithms (Martínez and Virtriá 2000;Pernkopf and Bouchaffra 2005), we perform the GA-EM using a single mixture model and treat the mixture model components as the individuals for GA.…”

Section: Mixture Model Estimationmentioning

confidence: 99%

Dual stream speech recognition using articulatory syllable models

Puurula

Compernolle

2010

Int J Speech Technol

View full text Add to dashboard Cite

Recent theoretical developments in neuroscience suggest that sublexical speech processing occurs via two parallel processing pathways. According to this Dual Stream Model of Speech Processing speech is processed both as sequences of speech sounds and articulations. We attempt to revise the "beads-on-a-string" paradigm of Hidden Markov Models in Automatic Speech Recognition (ASR) by implementing a system for dual stream speech recognition. A baseline recognition system is enhanced by modeling of articulations as sequences of syllables. An efficient and complementary model to HMMs is developed by formulating Dynamic Time Warping (DTW) as a probabilistic model. The DTW Model (DTWM) is improved by enriching syllable templates with constrained covariance matrices, data imputation, clustering and mixture modeling. The resulting dual stream system is evaluated on the N-Best Southern Dutch Broadcast News benchmark. Promising results are obtained for DTWM classification and ASR tests. We provide a discussion on the remaining problems in implementing dual stream speech recognition.

show abstract

“…More generic techniques like deterministic annealing [31], [35] and genetic algorithms [23], [17] have been applied to obtain a good set of parameters. Although these techniques have asymptotic guarantees, they are very time consuming and, hence, cannot be used in most practical applications.…”

Section: Relevant Backgroundmentioning

confidence: 99%

TRUST-TECH-Based Expectation Maximization for Learning Finite Mixture Models

Reddy

Chiang

Rajaratnam

2008

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Abstract-The Expectation Maximization (EM) algorithm is widely used for learning finite mixture models despite its greedy nature. Most popular model-based clustering techniques might yield poor clusters if the parameters are not initialized properly. To reduce the sensitivity of initial points, a novel algorithm for learning mixture models from multivariate data is introduced in this paper. The proposed algorithm takes advantage of TRUST-TECH (TRansformation Under STability-reTaining Equilibria CHaracterization) to compute neighborhood local maxima on the likelihood surface using stability regions. Basically, our method coalesces the advantages of the traditional EM with that of the dynamic and geometric characteristics of the stability regions of the corresponding nonlinear dynamical system of the log-likelihood function. Two phases, namely, the EM phase and the stability region phase, are repeated alternatively in the parameter space to achieve local maxima with improved likelihood values. The EM phase obtains the local maximum of the likelihood function and the stability region phase helps to escape out of the local maximum by moving toward the neighboring stability regions. Though applied to Gaussian mixtures in this paper, our technique can be easily generalized to any other parametric finite mixture model. The algorithm has been tested on both synthetic and real data sets and the improvements in the performance compared to other approaches are demonstrated. The robustness with respect to initialization is also illustrated experimentally.

show abstract

Learning mixture models using a genetic version of the EM algorithm

Cited by 36 publications

References 11 publications

Maximum likelihood estimation of Gaussian mixture models using stochastic search

Maximum likelihood estimation of Gaussian mixture models using stochastic search

Dual stream speech recognition using articulatory syllable models

TRUST-TECH-Based Expectation Maximization for Learning Finite Mixture Models

Contact Info

Product

Resources

About