Efficient parameter sampling for Markov jump processes

Abstract

Markov jump processes (MJPs) are continuous-time stochastic processes widely used in a variety of applied disciplines. Inference for MJPs typically proceeds via Markov chain Monte Carlo, the state-of-the-art being a uniformization-based auxiliary variable Gibbs sampler. This was designed for situations where the MJP parameters are known, and Bayesian inference over unknown parameters is typically carried out by incorporating it into a larger Gibbs sampler. This strategy of sampling parameters given path, and path given parameters can result in poor Markov chain mixing. In this work, we propose a simple and elegant algorithm to address this problem. Our scheme brings Metropolis-Hastings approaches for discrete-time hidden Markov models to the continuous-time setting, resulting in a complete and clean recipe for parameter and path inference in MJPs. In our experiments, we demonstrate superior performance over Gibbs sampling, as well as another popular approach, particle MCMC. We also show our sampler inherits geometric mixing from an 'ideal' sampler that operates without computational constraints.

Scalable Bayesian Inference for Population Markov Jump Processes

“…This paper has presented a novel and comprehensive framework for the design of scalable dataaugmentation procedures, suitable for use within exact Bayesian inferential tasks, and applicable to birth-death, epidemic or predator-prey systems, to name only a few. The need for auxiliary-variable augmentation designs as presented here is justified by the limitations in existing state-of-the-art uniformization-based approaches (see Hobolth and Stone, 2009;Teh, 2012, 2013;Miasojedow and Niemiro, 2015;Georgoulas et al, 2017;Zhang and Rao, 2018, and references therein), which are inefficient, unadaptable or unusable with mid-sized or large population systems, often associated with multiple types of observational data.…”

“…In order to preserve asymptotic exactness, without imposing artificial bounds on population levels, sequential particle procedures may be used to target sequences of states in x (Miasojedow and Niemiro, 2015) (subject to particle degeneracy), or arbitrary random truncations imposed over explorable spaces of paths (Georgoulas et al, 2017) (requiring costly Metropolis-Hastings (M-H) acceptance steps to overcome induced bias). Most recent advances towards efficient algorithmic constructions (see Zhang and Rao, 2018) involve updating parameters λ within forward-backward procedures for ( t, x), which works reportedly well with small MJP systems.…”

“…Population Markov jump processes (MJPs) are stochastic processes whose dynamics underpin many observable phenomena, in diverse fields such as stochastic epidemic (O'Neill and Roberts, 1999), immigration-death systems (Cappé et al, 2003;Zhang and Rao, 2018), chemical/molecular models (Hobolth and Stone, 2009;Georgoulas et al, 2017) or queueing systems (Sutton and Jordan, 2011;, to name only a few. In this paper, we present a novel and general framework for designing scalable auxiliary-variable data-augmentation algorithms, which allow for exact (in Monte Carlo sense) Bayesian inference for MJPs.…”

Stochastic Epidemic Models inference and diagnosis with Poisson Random Measure Data Augmentation

“…Different methods have already been proposed for the inference of Markov jump processes, some specific to epidemic models [6,11,12,13,14], and some more general [15,16,17]. In this article, we present PRM augmentation, used with Metropolis-Hastings sampling [18,19], and discuss its advantages and drawbacks, in terms of ease of use, speed, and insights, and illustrate this on some synthetic and real datasets.…”

“…PRM augmentation relies on a generalization of uniformization [20,15,17] that is directly applicable to any Markov jump process, with writing the Stochastic Differential Equation (SDE) being the only mathematical work needed. As such, it is a simulation-based method, which can be used easily to compare different models.…”