NuZZ: numerical Zig-Zag sampling for general models

Pagani, Filippo; Chevallier, Augustin; Power, Sam; House, Thomas; Cotter, Simon L.

doi:10.48550/arxiv.2003.03636

Cited by 3 publications

(9 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To our knowledge, there are only two papers that provide a general tool to draw samples using PDMPs requiring only the evaluation of the gradient of the target density. The Numeric Zig-Zag (NuZZ) (Pagani et al 2022) uses numerical integration to simulate the next switching event by time-scale transformation. The numeric integrator requires the evaluation of the rate λ(t) for a grid of values for t (from 7 to 14 points), and it is computed at each iteration of a root-finding method that derives the switching time (τ in Equation 7).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Zig-Zag sampling in practice

2022

View full text Add to dashboard Cite

Novel Monte Carlo methods to generate samples from a target distribution, such as a posterior from a Bayesian analysis, have rapidly expanded in the past decade. Algorithms based on Piecewise Deterministic Markov Processes (PDMPs), non-reversible continuous-time processes, are developing into their own research branch, thanks their important properties (e.g., super-efficiency). Nevertheless, practice has not caught up with the theory in this field, and the use of PDMPs to solve applied problems is not widespread. This might be due, firstly, to several implementational challenges that PDMP-based samplers present with and, secondly, to the lack of papers that showcase the methods and implementations in applied settings. Here, we address both these issues using one of the most promising PDMPs, the Zig-Zag sampler, as an archetypal example. After an explanation of the key elements of the Zig-Zag sampler, its implementation challenges are exposed and addressed. Specifically, the formulation of an algorithm that draws samples from a target distribution of interest is provided. Notably, the only requirement of the algorithm is a closed-form differentiable function to evaluate the log-target density of interest, and, unlike previous implementations, no further information on the target is needed. The performance of the algorithm is evaluated against canonical Hamiltonian Monte Carlo, and it is proven to be competitive, in simulation and real-data settings. Lastly, we demonstrate that the super-efficiency property, i.e. the ability to draw one independent sample at a lesser cost than evaluating the likelihood of all the data, can be obtained in practice.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Nevertheless, their use is not yet widespread, and very few papers use PDMP-based algorithms to address Bayesian estimation problems (Chevallier et al 2020;Koskela 2022). Even fewer papers attempt to implement PDMP-based algorithm in a general form (Bertazzi et al 2021;Pagani et al 2022), unfortunately they don't retain exactness.…”

Section: And Jagsmentioning

confidence: 99%

Automatic Zig-Zag sampling in practice

2022

View full text Add to dashboard Cite

show abstract

“…To our knowledge, there are only two papers that provide a general tool to draw samples using PDMPs requiring only the evaluation of the gradient of the target density. The Numeric Zig-Zag (NuZZ) (Pagani et al, 2022) uses numerical integration to simulate the next switching event by time-scale transformation. Another simulation scheme for PDMPs is proposed in Bertazzi, Bierkens, and Dobson, 2021, which solves the same problem by exploiting Euler approximations of the switching rate, abandoning exactness for the sake of generalizability.…”

Section: Discussionmentioning

confidence: 99%

Automatic Zig-Zag sampling in practice

Corbella¹,

Spencer²,

Roberts³

2022

Preprint

View full text Add to dashboard Cite

Novel Monte Carlo methods to generate samples from a target distribution, such as a posterior from a Bayesian analysis, have rapidly expanded in the past decade. Algorithms based on Piecewise Deterministic Markov Processes (PDMPs), non-reversible continuous-time processes, are developing into their own research branch, thanks their important properties (e.g., correct invariant distribution, ergodicity, and super-efficiency).Nevertheless, practice has not caught up with the theory in this field, and the use of PDMPs to solve applied problems is not widespread. This might be due, firstly, to several implementational challenges that PDMP-based samplers present with and, secondly, to the lack of papers that showcase the methods and implementations in applied settings. Here, we address both these issues using one of the most promising PDMPs, the Zig-Zag sampler, as an archetypal example.After an explanation of the key elements of the Zig-Zag sampler, its implementation challenges are exposed and addressed. Specifically, the formulation of an algorithm that draws samples from a target distribution of interest is provided. Notably, the only requirement of the algorithm is a closed-form function to evaluate the target density of interest, and, unlike previous implementations, no further information on the target is needed.The performance of the algorithm is evaluated against another gradient-based sampler, and it is proven to be competitive, in simulation and real-data settings. Lastly, we demonstrate that the super-efficiency property, i.e. the ability to draw one independent sample at a lesser cost than evaluating the likelihood of all the data, can be obtained in practice.

show abstract

“…In that case, the random event times have to be approximated even if the ODE can be solved exactly. This question has recently been addressed in [3,38,14] with three different schemes. Here, rather than designing an ad hoc numerical schemes, we work in the general framework of splitting schemes, which are widely used for e.g.…”

Section: Introductionmentioning

confidence: 99%

“…Moreover, in this work we describe how to remove the bias introduced by our approximation with a non-reversible Metropolis-Hastings step. Two other works ( [38] and [14]) focus on approximate simulation of the Zig-Zag sampler, which is one of our two main examples. In [38] the authors suggest to approximate event times by using numerical approximations of the integral of the rates along the dynamics (2), as well as a root finding algorithm.…”

Section: Introductionmentioning

confidence: 99%

Splitting schemes for second order approximations of piecewise-deterministic Markov processes

Bertazzi¹,

Dobson²,

Monmarché³

2023

Preprint

View full text Add to dashboard Cite

Numerical approximations of piecewise-deterministic Markov processes based on splitting schemes are introduced, together with their Metropolis-adjusted versions. The unadjusted schemes are shown to have a weak error of order two in the step size in a general framework. Focusing then on unadjusted schemes based on the Bouncy Particle and Zig-Zag samplers, we provide conditions ensuring ergodicity and consider the expansion of the invariant measure in terms of the step size. The dependency of the leading term in this expansion in terms of the refreshment rate, depending of the splitting order, is analyzed. Finally, we present numerical experiments on Gaussian targets, a Bayesian imaging inverse problem and a system of interacting particles.

show abstract

NuZZ: numerical Zig-Zag sampling for general models

Cited by 3 publications

References 0 publications

Automatic Zig-Zag sampling in practice

Automatic Zig-Zag sampling in practice

Automatic Zig-Zag sampling in practice

Splitting schemes for second order approximations of piecewise-deterministic Markov processes

Contact Info

Product

Resources

About