Robust Sequential Data Modeling Using an Outlier Tolerant Hidden Markov Model

Abstract

Hidden Markov (chain) models using finite Gaussian mixture models as their hidden state distributions have been successfully applied in sequential data modeling and classification applications. Nevertheless, Gaussian mixture models are well known to be highly intolerant to the presence of untypical data within the fitting data sets used for their estimation. Finite Student's t-mixture models have recently emerged as a heavier-tailed, robust alternative to Gaussian mixture models, overcoming these hurdles. To exploit these merits of Student's t-mixture models in the context of a sequential data modeling setting, we introduce, in this paper, a novel hidden Markov model where the hidden state distributions are considered to be finite mixtures of multivariate Student's t-densities. We derive an algorithm for the model parameters estimation under a maximum likelihood framework, assuming full, diagonal, and factor-analyzed covariance matrices. The advantages of the proposed model over conventional approaches are experimentally demonstrated through a series of sequential data modeling applications.

A variational Bayesian methodology for hidden Markov models utilizing Student's-t mixtures

“…the resulting Student's-t HMM has been treated under a maximum-likelihood framework using the EM algorithm [4].…”

“…Note that exactly the same procedure would be employed to obtain the likelihood of a sequence with respect to an SHMM trained using the EM algorithm [4].…”

“…To tackle these issues, we have proposed in [4] a novel form of continuous HMMs where the hidden state distributions are modeled using finite mixtures of multivariate Student's-t densities. The multivariate Student's-t distribuEmail addresses: soteri0s@me.com (Sotirios P. Chatzis), dkosmo@iit.demokritos.gr (Dimitrios I. Kosmopoulos).…”

“…The so-obtained Student's-t hidden Markov model (SHMM) has been considered in [4] under the ML paradigm using the EM algorithm; as it has been shown, the SHMM provides an effective, computationally efficient and application-independent means for outlier tolerant representation and classification of sequential data by means of continuous HMMs.…”

The Infinite-Order Conditional Random Field Model for Sequential Data Modeling

“…This can be effected in a computationally efficient manner using the familiar forward-backward algorithm [32], [7], widely known from the HMM literature. Indeed, as discussed, e.g., in [36], it is easy to show that…”