Fast mixing of Metropolized Hamiltonian Monte Carlo: Benefits of multi-step gradients

Chen, Yuansi; Dwivedi, Raaz; Wainwright, Martin J.; Yu, Bin

doi:10.48550/arxiv.1905.12247

Cited by 6 publications

(17 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Proof. We generalise the arguments from [14], Lemma 7. Proceeding by induction over n, we have for the case n = 1, for any v ∈ R d , that DT 1 (v) = hC and S 1 (v) = 1 h C −1 q 0 − h 2 C ∇U (q 0 ) with derivative of zero.…”

Section: B Proof Of Lemmamentioning

confidence: 96%

“…the inverse of the spectral gap, grows linear in κ, assuming the integration time is set to T = 1 2 √ m2 . [14] establish non-asymptotic upper bounds on the mixing time using a leap-frog integrator where the step size h and the number L of steps depends explicitly on m 1 and m 2 . Convergence guarantees are established using conductance profiles by obtaining (i) a high probability lower bound on the acceptance rate and (ii) an overlap bound, that is a lower bound on the KL-divergence between the HMC proposal densities at the starting positions q 0 and q 0 , whenever q 0 is close to q 0 .…”

Section: Related Workmentioning

confidence: 99%

“…see also [42,21,14] for the special case with an identity mass matrix. Observe that for L = 1 leap-frog steps, this reduces to a MALA proposal with preconditioning matrix M −1 .…”

Section: Marginal Proposal Entropymentioning

confidence: 99%

“…Suppose that the current state of the Markov chain is q. We resample the momentum v ∼ N (0, I) and aim to solve (14) by taking gradients of the penalised loss function…”

Section: Adaptation With a Generalised Speed Measurementioning

confidence: 99%

“…For completeness, we provide a proof of the reparameterization ( 3) and ( 4) of the L-th step position q L and momentum p L using the velocity v that relates to the initial momentum p 0 ∼ N (0, M ) via p 0 = C − v. Such representations with an identity mass matrix have been used previously in [42,21,14].…”

Section: Funding Transparency Statementmentioning

confidence: 99%

See 4 more Smart Citations

Entropy-based adaptive Hamiltonian Monte Carlo

Hirt¹,

Titsias²,

Δελλαπόρτας³

2021

Preprint

View full text Add to dashboard Cite

Hamiltonian Monte Carlo (HMC) is a popular Markov Chain Monte Carlo (MCMC) algorithm to sample from an unnormalized probability distribution. A leapfrog integrator is commonly used to implement HMC in practice, but its performance can be sensitive to the choice of mass matrix used therein. We develop a gradient-based algorithm that allows for the adaptation of the mass matrix by encouraging the leapfrog integrator to have high acceptance rates while also exploring all dimensions jointly. In contrast to previous work that adapt the hyperparameters of HMC using some form of expected squared jumping distance, the adaptation strategy suggested here aims to increase sampling efficiency by maximizing an approximation of the proposal entropy. We illustrate that using multiple gradients in the HMC proposal can be beneficial compared to a single gradientstep in Metropolis-adjusted Langevin proposals. Empirical evidence suggests that the adaptation method can outperform different versions of HMC schemes by adjusting the mass matrix to the geometry of the target distribution and by providing some control on the integration time.35th Conference on Neural Information Processing Systems (NeurIPS 2021).

show abstract

Section: B Proof Of Lemmamentioning

confidence: 96%

Section: Related Workmentioning

confidence: 99%

“…see also [42,21,14] for the special case with an identity mass matrix. Observe that for L = 1 leap-frog steps, this reduces to a MALA proposal with preconditioning matrix M −1 .…”

Section: Marginal Proposal Entropymentioning

confidence: 99%

“…Suppose that the current state of the Markov chain is q. We resample the momentum v ∼ N (0, I) and aim to solve (14) by taking gradients of the penalised loss function…”

Section: Adaptation With a Generalised Speed Measurementioning

confidence: 99%

Section: Funding Transparency Statementmentioning

confidence: 99%

See 3 more Smart Citations

Entropy-based adaptive Hamiltonian Monte Carlo

Hirt¹,

Titsias²,

Δελλαπόρτας³

2021

Preprint

View full text Add to dashboard Cite

show abstract

Log-concave sampling: Metropolis-Hastings algorithms are fast

Dwivedi,

Chen,

Wainwright

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

We study the problem of sampling from a strongly log-concave density supported on R d , and prove a non-asymptotic upper bound on the mixing time of the Metropolis-adjusted Langevin algorithm (MALA). The method draws samples by simulating a Markov chain obtained from the discretization of an appropriate Langevin diffusion, combined with an accept-reject step. Relative to known guarantees for the unadjusted Langevin algorithm (ULA), our bounds show that the use of an accept-reject step in MALA leads to an exponentially improved dependence on the error-tolerance. Concretely, in order to obtain samples with TV error at most δ for a density with condition number κ, we show that MALA requires O κd log(1/δ) steps from a warm start, as compared to the O κ 2 d/δ 2 steps established in past work on ULA. We also demonstrate the gains of a modified version of MALA over ULA for weakly log-concave densities. Furthermore, we derive mixing time bounds for the Metropolized random walk (MRW) and obtain O(κ) mixing time slower than MALA. We provide numerical examples that support our theoretical findings, and demonstrate the benefits of Metropolis-Hastings adjustment for Langevin-type sampling algorithms.

show abstract

Estimating Normalizing Constants for Log-Concave Distributions: Algorithms and Lower Bounds

Ge¹,

Lee²,

Lu³

2019

Preprint

View full text Add to dashboard Cite

Estimating the normalizing constant of an unnormalized probability distribution has important applications in computer science, statistical physics, machine learning, and statistics. In this work, we consider the problem of estimating the normalizing constant Z = R d e −f (x) dx to within a multiplication factor of 1 ± ε for a µ-strongly convex and L-smooth function f , given query access to f (x) and ∇f (x). We give both algorithms and lowerbounds for this problem. Using an annealing algorithm combined with a multilevel Monte Carlo method based on underdamped Langevin dynamics, we show that O d 4/3 κ+d 7/6 κ 7/6 ε 2 queries to ∇f are sufficient, where κ = L/µ is the condition number. Moreover, we provide an information theoretic lowerbound, showing that at least d 1−o(1) ε 2−o(1) queries are necessary. This provides a first nontrivial lowerbound for the problem.

show abstract

Fast mixing of Metropolized Hamiltonian Monte Carlo: Benefits of multi-step gradients

Cited by 6 publications

References 47 publications

Entropy-based adaptive Hamiltonian Monte Carlo

Entropy-based adaptive Hamiltonian Monte Carlo

Log-concave sampling: Metropolis-Hastings algorithms are fast

Estimating Normalizing Constants for Log-Concave Distributions: Algorithms and Lower Bounds

Contact Info

Product

Resources

About