Standard MCMC methods can scale poorly to big data settings due to the need to evaluate the likelihood at each iteration. There have been a number of approximate MCMC algorithms that use sub-sampling ideas to reduce this computational burden, but with the drawback that these algorithms no longer target the true posterior distribution. We introduce a new family of Monte Carlo methods based upon a multi-dimensional version of the Zig-Zag process of Bierkens and Roberts (2017), a continuous time piecewise deterministic Markov process. While traditional MCMC methods are reversible by construction (a property which is known to inhibit rapid convergence) the Zig-Zag process offers a flexible non-reversible alternative which we observe to often have favourable convergence properties. We show how the Zig-Zag process can be simulated without discretisation error, and give conditions for the process to be ergodic. Most importantly, we introduce a sub-sampling version of the Zig-Zag process that is an example of an exact approximate scheme, i.e. the resulting approximate process still has the posterior as its stationary distribution. Furthermore, if we use a control-variate idea to reduce the variance of our unbiased estimator, then the Zig-Zag process can be super-efficient: after an initial pre-processing step, essentially independent samples from the posterior distribution are obtained at a computational cost which does not depend on the size of the data. * The authors acknowledge the EPSRC for support under grants EP/D002060/1 (CRiSM) and EP/K014463/1 (iLike).MSC 2010 subject classifications: Primary 65C60; secondary 65C05, 62F15, 60J25

In Turitsyn, Chertkov and Vucelja (2011) a non-reversible Markov Chain Monte Carlo (MCMC) method on an augmented state space was introduced, here referred to as Lifted Metropolis-Hastings (LMH). A scaling limit of the magnetization process in the Curie-Weiss model is derived for LMH, as well as for Metropolis-Hastings (MH). The required jump rate in the high (supercritical) temperature regime equals n 1/2 for LMH, which should be compared to n for MH. At the critical temperature the required jump rate equals n 3/4 for LMH and n 3/2 for MH, in agreement with experimental results of Turitsyn, Chertkov and Vucelja (2011). The scaling limit of LMH turns out to be a non-reversible piecewise deterministic exponentially ergodic 'zig-zag' Markov process. Primary 60F05; secondary 65C05. Keywords and phrases: weak convergence, Markov chain Monte Carlo, piecewise deterministic Markov process, phase transition, exponential ergodicity. * Both authors would like to thank the EPSRC for support under grants EP/D002060/1 (CRiSM) and EP/K014463/1 (iLike) 1 imsart-generic ver. 2011/11/15 file: spin.tex date: June 3, 2016 J. Bierkens and G. Roberts/Piecewise deterministic scaling limit of Lifted Metropolis-Hastings 2 'lifting approach' is not the only way of obtaining non-reversible Markov processes. For interesting approaches to constructing and analyzing the benefits of non-reversible Markov processes, see e.g. Hwang, Duncan, Lelièvre and Pavliotis (2015).It is the goal of this paper to shed light on the general theory of lifted non-reversible Markov chains, and in particular on the recent experimental result of Turitsyn, Chertkov and Vucelja (2011) mentioned above. This is achieved by obtaining a scaling limit of Lifted Metropolis-Hastings, in its application to the Curie-Weiss model. This scaling limit may be compared to a similar scaling limit for (classical) Metropolis-Hastings.Initiated by Roberts, Gelman and Gilks (1997), a large amount of understanding of particular Markov Chain Monte Carlo (MCMC) algorithms has been obtained by identifying a suitable diffusion limit: Given a sequence of Markov chains of increasing size or dimensionality n, a suitable scaling of the state space and of the amount of steps per unit time interval is determined. As n tends towards infinity, the scaled Markov process converges (in the sense of weak convergence on Skorohod path space) to a diffusion process, which is often of an elementary nature. In particular the required number of Markov chain transitions per unit time interval as a function of n provides a fundamental measure of the speed of the Markov chain.The Curie-Weiss model is an exchangeable probability distribution on {−1, 1} n which depends on two parameters, the 'external field' h, and the 'inverse temperature' β (which describes interactions between components). At inverse temperature β = 1 the model undergoes a phase transition. This results in differences in behaviour for β < 1, β = 1 and β > 1, and we shall analyse the behaviour of both standard Metropolis-Hastings and Lifted Me...

The zigzag process is a variant of the telegraph process with position dependent switching intensities. A characterization of the L 2spectrum for the generator of the one-dimensional zigzag process is obtained in the case where the marginal stationary distribution on R is unimodal and the refreshment intensity is zero. Sufficient conditions are obtained for a spectral mapping theorem, mapping the spectrum of the generator to the spectrum of the corresponding Markov semigroup. Furthermore results are obtained for symmetric stationary distributions and for perturbations of the spectrum, in particular for the case of a non-zero refreshment intensity.Keywords and phrases: spectral theory, non-reversible Markov process, Markov semigroup, zigzag process, exponential ergodicity, perturbation theory. * This work is part of the research programme 'Zig-zagging through computational barriers' with project number 016.Vidi.189.043, which is financed by the Netherlands Organisation for Scientific Research (NWO).

Recently there have been exciting developments in Monte Carlo methods, with the development of new MCMC and sequential Monte Carlo (SMC) algorithms which are based on continuous-time, rather than discrete-time, Markov processes. This has led to some fundamentally new Monte Carlo algorithms which can be used to sample from, say, a posterior distribution. Interestingly, continuous-time algorithms seem particularly well suited to Bayesian analysis in big-data settings as they need only access a small sub-set of data points at each iteration, and yet are still guaranteed to target the true posterior distribution. Whilst continuous-time MCMC and SMC methods have been developed independently we show here that they are related by the fact that both involve simulating a piecewise deterministic Markov process. Furthermore we show that the methods developed to date are just specific cases of a potentially much wider class of continuous-time Monte Carlo algorithms. We give an informal introduction to piecewise deterministic Markov processes, covering the aspects relevant to these new Monte Carlo algorithms, with a view to making the development of new continuous-time Monte Carlo more accessible. We focus on how and why sub-sampling ideas can be used with these algorithms, and aim to give insight into how these new algorithms can be implemented, and what are some of the issues that affect their efficiency.

The classical Metropolis-Hastings (MH) algorithm can be extended to generate non-reversible Markov chains. This is achieved by means of a modification of the acceptance probability, using the notion of vorticity matrix. The resulting Markov chain is non-reversible. Results from the literature on asymptotic variance, large deviations theory and mixing time are mentioned, and in the case of a large deviations result, adapted, to explain how non-reversible Markov chains have favorable properties in these respects. We provide an application of NRMH in a continuous setting by developing the necessary theory and applying, as first examples, the theory to Gaussian distributions in three and nine dimensions. The empirical autocorrelation and estimated asymptotic variance for NRMH applied to these examples show significant improvement compared to MH with identical stepsize.