Speeding Up Bayesian HMM by the Four Russians Method

Abstract

Bayesian computations with Hidden Markov Models (HMMs) are often avoided in practice. Instead, due to reduced running time, point estimates -maximum likelihood (ML) or maximum a posterior (MAP) -are obtained and observation sequences are segmented based on the Viterbi path, even though the lack of accuracy and dependency on starting points of the local optimization are well known. We propose a method to speed-up Bayesian computations which addresses this problem for regular and time-dependent HMMs with discrete observations. In particular, we show that by exploiting sequence repetitions, using the four Russians method, and the conditional dependency structure, it is possible to achieve a Θ(log T ) speed-up, where T is the length of the observation sequence. Our experimental results on identification of segments of homogeneous nucleic acid composition, known as the DNA segmentation problem, show that the speed-up is also observed in practice. Availability: An implementation of our method will be available as part of the open source GHMM library from http://ghmm.org.

Compressed computations using wavelets for hidden Markov models with continuous observations

“…In recent years, compression has become an effective computational building block to accelerate algorithms in response to ever-increasing data set sizes across a wide range of fields. The main idea is often to avoid re-computation for identical patterns in the data, as in the case for Hidden Markov Models (HMM) with discrete-valued observations whether for the frequentist classical three algorithms [9] or Gibbs sampling for Bayesian HMM [12].…”

“…Also for discrete observations, Mahmud et al [ 12 ] used transition-emission operators to substantially accelerate Forward-Backward Gibbs (FBG) sampling [ 13 ], a rapidly converging sampler for computations with fully Bayesian HMM, the fourth problem for HMM. There, marginal state probabilities conditioned on the data provides a robust alternative to locally optimal ML estimation followed by Viterbi-computation.…”

Fast Bayesian Inference of Copy Number Variants using Hidden Markov Models with Wavelet Compression

“…Bayesian methods have obtained a reputation of requiring enormous computational effort and being difficult to use, for the expert knowledge required for choosing prior distributions. It has also been recognized [60,62,73] that they are very powerful and accurate, leading to improved, high-quality results and providing, in the form of posterior distributions, an accurate measure of uncertainty in results. Nevertheless, it is not surprising that a hundred times larger effort in computational effort alone prevented wide-spread use.…”

“…Though there are several schemes available to sample q, [58] has argued strongly in favor of Forward-Backward sampling [57], which yields Forward-Backward Gibbs sampling (FBG) above. Variations of this have been implemented for segmentation of aCGH data before [60,62,78]. However, since in each iteration a quadratic number of terms has to be calculated at each position to obtain the forward variables, and a state has to be sampled at each position in the backward step, this method is still expensive for large data.…”

“…Though there are several sampling schemes available, [58] has argued strongly in favor of Forward-Backward Gibbs sampling (FBG) [57]. Variations of this have been implemented for segmentation of aCGH data before [60,62,73]. However, sampling-based Bayesian methods such as FBG are computationally expensive.…”