When memory pays: Discord in hidden Markov models

Abstract

When is keeping a memory of observations worthwhile? We use hidden Markov models to look at phase transitions that emerge when comparing state estimates in systems with discrete states and noisy observations. We infer the underlying state of the hidden Markov models from the observations in two ways: through naive observations, which take into account only the current observation, and through Bayesian filtering, which takes the history of observations into account. Defining a discord order parameter to distinguish between the different state estimates, we explore hidden Markov models with various numbers of states and symbols and varying transition-matrix symmetry. All behave similarly. We calculate analytically the critical point where keeping a memory of observations starts to pay off. A mapping between hidden Markov models and Ising models gives added insight into the associated phase transitions.

Phase Transition in Protocols Minimizing Work Fluctuations

“…The trade-offs between the two, captured by the Pareto front, have the structure of a first-order phase transition with phase-coexistence protocols interpolating between the two phases. Such a phase transition may be a common feature of optimization problems: they occur in optimal complex networks [57,58], statistical in-ference [59]; and similar phenomena were observed in a quantum control problem with varying constraints [60] and in the utilization of memory in an information engine [61][62][63]. Finally, we observed that the minimum work fluctuation and the phase coexistence protocols do not become quasistatic even for very long protocols and are thus not accessible by a linear theory.…”

Phase transition for parameter learning of Hidden Markov Models

“…For example, the entropy of a binary HMM was calculated [26] by a mapping to a one-dimensional Ising model. Lathouwers and Bechhoefer have investigated [27] transitions with respect to whether the reconstruction of a hidden sequence is possible or not depending on whether data can be kept in memory. Allahverdyan and Galastyan have investigated [28,29] the maximum a posteriori (Viterbi) sequence as a function of a noise parameter and found transitions between regions where almost full sequence reconstruction is possible and regions where not.…”

Optimal work extraction and the minimum description length principle

“…Recent progress within the emerging field of stochastic thermodynamics has led to a deeper understanding of the relationship between information and entropy at mesoscopic scales, leading to e.g., the refinement of the second law of thermodynamics for systems with feedback control [7]. These works study fundamental thermodynamic bounds governing processes that convert information into work and/or heat, and vice versa, both in theory [5,[8][9][10][11][12][13][14][15][16][17][18][19], and experiment [20][21][22][23][24][25][26][27][28].…”