Adapting Hidden Markov Models for Online Learning

Abstract

© 2015 The Authors. Published by Elsevier B.V.In modern computer systems, the intermittent behaviour of infrequent, additional loads affects performance. Often, representative traces of storage disks or remote servers can be scarce and obtaining real data is sometimes expensive. Therefore, stochastic models, through simulation and profiling, provide cheaper, effective solutions, where input model parameters are obtained. A typical example is the Markov-modulated Poisson process (MMPP), which can have its time index discretised to form a hidden Markov model (HMM). These models have been successful in capturing bursty behaviour and cyclic patterns of I/O operations and Internet traffic, using underlying properties of the discrete (or continuous) Markov chain. However, learning on such models can be cumbersome in terms of complexity through re-training on data sets. Thus, we provide an online learning HMM (OnlineHMM), which is composed of two existing variations of HMMs: first, a sliding HMM using a moving average technique to update its parameters on-the-fly and, secondly, a multi-input HMM capable of training on multiple discrete traces simultaneously. The OnlineHMM reduces data processing times significantly and thence synthetic workloads become computationally more cost effective. We measure the accuracy of reproducing representative traces through comparisons of moments and autocorrelation on original data points and HMM-generated synthetic traces. We present, analytically, the training steps saved through the OnlineHMMs adapted Baum-Welch algorithm and obtain, through simulation, mean waiting times of a queueing model. Finally, we conclude our work and offer model extensions for the future

Higher response time moments for M/M/1 discriminatory processor sharing queues

“…Discretized MMPPs (or hidden Markov models) replicate the burstiness of TCP packet traces, which can be clustered in groups, and, hence, allow model parameters to converge on multiple traces simultaneously at reduced computational complexity [10]. Further, arrival parameters of queueing models can be updated incrementally via online EM learning algorithms [2,6,14], which are suitable for live systems.…”

“…The GLD "quality-of-fit" is ascertained only through a goodness-of-fit test as closed-form solutions do not exist for the lambda parameters. With response time moments obtained from variations of equations (4), (5), (6) and the algorithm from figure 1 (with different values for α priority weights), one can easily parametrize the GLD and obtain distributions. For our results, we compare response time moments from our analytical approximations with simulations on two different case studies.…”

A computational approach to the study of interactions between proteins and miR10-b, miR-335, and miR-21 involved in breast cancer

“…In order to confirm our results of docking we used the HMM prediction method to compare its results with the docking results. Thus, in the present study, we used the aPPRove algorithm based on HMM [34] to validate the results using the HMM coupled to Bayesian models. The transition matrix was defined in order to study the transition probability among states.…”

Design Strategy Transfer in Cognitively-Inspired Agents

“…This online learning method is based on the method previously utilized in CISAT [23] and has been adapted here to be used with Hidden Markov Models to maintain uniformity when comparing different operation selection models in CISAT. Standard algorithms like Baum-Welch [33], are not stable for online learning in their original formulation [34] and significant advancements [35,36] need to be implemented for application in an online setting, an area to be explored in future work.…”