Speed Up Zig-Zag

Abstract

Zig-Zag is Piecewise Deterministic Markov Process, efficiently used for simulation in an MCMC setting. As we show in this article, it fails to be exponentially ergodic on heavy tailed target distributions. We introduce an extension of the Zig-Zag process by allowing the process to move with a non-constant speed function s, depending on the current state of the process. We call this process Speed Up Zig-Zag (SUZZ). We provide conditions that guarantee stability properties for the SUZZ process, including non-explosivity, exponential ergodicity in heavy tailed targets and central limit theorem. Interestingly, we find that using speed functions that induce explosive deterministic dynamics may lead to stable algorithms that can even mix faster. We further discuss the choice of an efficient speed function by providing an efficiency criterion for the one-dimensional process and we support our findings with simulation results.

A note on the polynomial ergodicity of the one-dimensional Zig-Zag process

“…However, a natural analogue of this modification is to allow the Zig-Zag to speed up and move faster in areas of lower density. This idea is further discussed in [23] and proved to be able to provide exponentially ergodic algorithms even on heavy-tailed targets, which can outperform the simple Zig-Zag in the sense of having better effective sample size per number of likelihood evaluations.…”

“…In [10] Bierkens, Roberts, and Zitt proved ergodicity and exponential ergodicity of the Zig-Zag process in arbitrary dimension, but a crucial assumption required for exponential ergodicity in their work is that the target density has exponential or lighter tails. In [23] Vasdekis and Roberts proved the converse result. The Zig-Zag sampler fails to be exponentially ergodic when the target distribution is heavy-tailed.…”

“…An immediate corollary of Theorem 1 is the following characterisation of the order of polynomial convergence of the one-dimensional Zig-Zag on Student distributions. For the following lower bound on the total variation distance from the invariant measure, we use a type of argument similar to the proof of Theorem 2.1 of [23], suggested to us by Professor Anthony Lee in a private communication.…”

Stereographic Markov Chain Monte Carlo

“…(1) This is the first known PDMP algorithm that is uniformly ergodic for a large family of target distributions including heavy-tailed targets. For comparison, the traditional BPS is only known to be geometrically ergodic under certain restrictive conditions on the target [DBCD19; DGM20] and is known not to be geometrically ergodic for any heavy-tailed target distribution [VR21b].…”

Continuously-Tempered PDMP Samplers

“…These PDMP samplers have a number of advantages, including non-reversible dynamics (which is known to improve mixing relative to reversible processes [9,1]), and the ability to reduce computation-per-iteration by either leveraging sparsity structure in the model [5,18] or using only sub-samples of the data to approximate the log-likelihood at each iteration (whilst still guaranteeing sampling from the target [2]). However, like other MCMC algorithms, particularly those that use gradient information, these PDMP samplers can struggle to mix for multi-modal target distributions, or for heavy-tailed targets [20].…”