A Generalized Framework For Kullback–Leibler Markov Aggregation

Abstract

This paper proposes an information-theoretic cost function for aggregating a Markov chain via a (possibly stochastic) mapping. The cost function is motivated by two objectives: 1) The process obtained by observing the Markov chain through the mapping should be close to a Markov chain, and 2) the aggregated Markov chain should retain as much of the temporal dependence structure of the original Markov chain as possible. We discuss properties of this parameterized cost function and show that it contains the cost functions previously proposed by Deng et al., Xu et al., and Geiger et al. as special cases. We moreover discuss these special cases providing a better understanding and highlighting potential shortcomings: For example, the cost function proposed by Geiger et al. is tightly connected to approximate probabilistic bisimulation, but leads to trivial solutions if optimized without regularization. We furthermore propose a simple heuristic to optimize our cost function for deterministic aggregations and illustrate its performance on a set of synthetic examples.

Co-Clustering via Information-Theoretic Markov Aggregation

“…with a alphabet Z by a Markov chain with a smaller alphabet Z, sacrificing model accuracy for a reduction in model complexity. Aggregation is usually performed by partitioning (i.e., clustering) the alphabet Z and defining a Markov chain on the partitioned alphabet Z. Information-theoretic cost functions for Markov aggregation had been proposed in, e.g., [20]- [22] and were recently unified in [23]. More generally, aggregations of dynamical systems that are not necessarily Markov were discussed in [24].…”

“…More generally, aggregations of dynamical systems that are not necessarily Markov were discussed in [24]. In contrast to [20]- [23], the cost functions proposed by [24] are task-specific in the sense that they aim to predict an observation based on Z t from the aggregated process.…”

“…The Markov aggregation problem is concerned with finding a function ζ: Z → Z, where typically |Z| |Z|, such that the reduced model captures relevant aspects of the original model. Specifically, the authors of [23] suggest trading between two different objectives: The objective to make the process {ζ(Z t )} as close to a Markov chain as possible, and the objective that {ζ(Z t )} preserves the temporal dependence structure of the original Markov chain {Z t }. They propose the following information-theoretic cost function for Markov aggregation: Definition 1 (Generalized Markov Aggregation [23]).…”

“…Specifically, the authors of [23] suggest trading between two different objectives: The objective to make the process {ζ(Z t )} as close to a Markov chain as possible, and the objective that {ζ(Z t )} preserves the temporal dependence structure of the original Markov chain {Z t }. They propose the following information-theoretic cost function for Markov aggregation: Definition 1 (Generalized Markov Aggregation [23]). Let {Z t } be a discrete-time, stationary Markov chain with alphabet Z and state transition matrix A, and suppose the set Z is given.…”

“…In particular, it can be optimized by adapting heuristics proposed for information-theoretic co-clustering by other authors (see Sections 2 and 5). Since our cost function is derived from the generalized information-theoretic Markov aggregation problem, co-clustering solutions can be obtained by employing the aggregation algorithm proposed in [23] taking into account the additional mutual exclusivity constraint. The algorithm is a simple sequential heuristic for minimizing L β , similar to the sequential IB algorithm proposed in [11] and the algorithm proposed by Dhillon et al for informationtheoretic co-clustering [2].…”