Switching Poisson Gamma Dynamical Systems

Abstract

<jats:p>We propose Switching Poisson gamma dynamical systems (SPGDS) to model sequentially observed multivariate count data. Different from previous models, SPGDS assigns its latent variables into mixture of gamma distributed parameters to model complex sequences and describe the nonlinear dynamics, meanwhile, capture various temporal dependencies. For efficient inference, we develop a scalable hybrid stochastic gradient-MCMC and switching recurrent autoencoding variational inference, which is scalable to large scale sequences and fast in out-of-sample prediction. Experiments on both unsupervised and supervised tasks demonstrate that the proposed model not only has excellent fitting and prediction performance on complex dynamic sequences, but also separates different dynamical patterns within them.</jats:p>

A Comparative Study of Gamma Markov Chains for Temporal Non-Negative Matrix Factorization

“…Note that recent works go beyond the Markovian assumption, i.e., assume dependency with multiple past time steps, and are labeled as "deep" (Gong and Huang, 2017;Guo et al, 2018). Several works (Févotte et al, 2013;Schein et al, 2016Schein et al, , 2019 assume that the transition distribution p(h n |h n−1 ) makes use of a transition matrix Π of size K × K to capture relationships between the different components. In this case, the distribution of h kn depends on a linear combination of all the components at the previous time step:…”

“…The authors set the value of α to 1 (although the same trick can be applied for any value of α). This model is also a particular case of the dynamical model of Schein et al (2016). It has since been used in the context of topic modeling (Acharya et al, 2018).…”

“…This model is a particular case of the dynamical model firstly introduced in Schein et al (2019). It cannot be rewritten as a multiplicative noise model.…”

“…The motivation behind the introduction of this model is once again computational: it leads to closed-form conditional distributions when considering a Poisson likelihood. As stated in Schein et al (2019), the auxiliary variable z n can actually be marginalized out, leading to the so-called randomized Gamma distribution of the first type (RG1), whose analytical expression makes use of modified Bessel functions.…”

“…Note that a Gibbs sampling procedure is proposed in Acharya et al (2015); Schein et al (2016), and as such the development of the MAP algorithm is novel.…”