Regular Perturbation of V-Geometrically Ergodic Markov Chains

Ferré, Déborah; Hervé, Loı̈c; Ledoux, James

doi:10.1017/s002190020001319x

Cited by 4 publications

(6 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lastly, our analysis relies heavily on the theory of ergodicity for MDPs. We build on the works of Mitrophanov (2005) which yields perturbation bounds on the state distribution, and subsequent improvements in the assumptions and condition numbers (Ferré, Hervé, and Ledoux 2013;Rudolf, Schweizer et al 2018;Mao and Song 2020).…”

Section: Related Work Optimization and Stochastic Approximationmentioning

confidence: 99%

Convergence and Optimality of Policy Gradient Methods in Weakly Smooth Settings

Zhang

Erdogdu

Garg

2022

AAAI

View full text Add to dashboard Cite

Policy gradient methods have been frequently applied to problems in control and reinforcement learning with great success, yet existing convergence analysis still relies on non-intuitive, impractical and often opaque conditions. In particular, existing rates are achieved in limited settings, under strict regularity conditions. In this work, we establish explicit convergence rates of policy gradient methods, extending the convergence regime to weakly smooth policy classes with L2 integrable gradient. We provide intuitive examples to illustrate the insight behind these new conditions. Notably, our analysis also shows that convergence rates are achievable for both the standard policy gradient and the natural policy gradient algorithms under these assumptions. Lastly we provide performance guarantees for the converged policies.

show abstract

Section: Related Work Optimization and Stochastic Approximationmentioning

confidence: 99%

Convergence and Optimality of Policy Gradient Methods in Weakly Smooth Settings

Zhang

Erdogdu

Garg

2022

AAAI

View full text Add to dashboard Cite

show abstract

“…The aim of this work is to use the results of [FHL13,HL14a,RS18] to investigate the robustness rst of the V a -geometrical ergodicity property (4), second of the stationary distribution π, third of the probability distribution of X n . This study is made with respect to parametric variations of both the function F and the p.d.…”

Section: Introductionmentioning

confidence: 99%

“…Under Assumption (H 1 ), do the perturbed transition kernels P θ satisfy the V a -geometrical ergodicity when θ is close to θ 0 ? In our context of parametric perturbation of IFS, these questions are addressed in the following theorem using the results of [FHL13,HL14a,RS18].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Robustness of iterated function systems of Lipschitz maps

Hervé

Ledoux

2023

J. Appl. Probab.

Self Cite

View full text Add to dashboard Cite

Let $\{X_n\}_{n\in{\mathbb{N}}}$ be an ${\mathbb{X}}$ -valued iterated function system (IFS) of Lipschitz maps defined as $X_0 \in {\mathbb{X}}$ and for $n\geq 1$ , $X_n\;:\!=\;F(X_{n-1},\vartheta_n)$ , where $\{\vartheta_n\}_{n \ge 1}$ are independent and identically distributed random variables with common probability distribution $\mathfrak{p}$ , $F(\cdot,\cdot)$ is Lipschitz continuous in the first variable, and $X_0$ is independent of $\{\vartheta_n\}_{n \ge 1}$ . Under parametric perturbation of both F and $\mathfrak{p}$ , we are interested in the robustness of the V-geometrical ergodicity property of $\{X_n\}_{n\in{\mathbb{N}}}$ , of its invariant probability measure, and finally of the probability distribution of $X_n$ . Specifically, we propose a pattern of assumptions for studying such robustness properties for an IFS. This pattern is implemented for the autoregressive processes with autoregressive conditional heteroscedastic errors, and for IFS under roundoff error or under thresholding/truncation. Moreover, we provide a general set of assumptions covering the classical Feller-type hypotheses for an IFS to be a V-geometrical ergodic process. An accurate bound for the rate of convergence is also provided.

show abstract

“…i=1 the Lagrange polynomials associated with the set {q Ferré et al (2013), combined with Theorems 3.14 and 3.16 of (Conn et al, 2009), can be used to obtain weaker sufficient conditions under which the condition in Lemma B.9…”

mentioning

confidence: 99%

Accelerating Asymptotically Exact MCMC for Computationally Intensive Models via Local Approximations

Conrad

Marzouk

Pillai

et al. 2016

Journal of the American Statistical Association

View full text Add to dashboard Cite

We construct a new framework for accelerating Markov chain Monte Carlo in posterior sampling problems where standard methods are limited by the computational cost of the likelihood, or of numerical models embedded therein. Our approach introduces local approximations of these models into the Metropolis-Hastings kernel, borrowing ideas from deterministic approximation theory, optimization, and experimental design. Previous efforts at integrating approximate models into inference typically sacrifice either the sampler's exactness or efficiency; our work seeks to address these limitations by exploiting useful convergence characteristics of local approximations. We prove the ergodicity of our approximate Markov chain, showing that it samples asymptotically from the exact posterior distribution of interest. We describe variations of the algorithm that employ either local polynomial approximations or local Gaussian process regressors. Our theoretical results reinforce the key observation underlying this paper: when the likelihood has some local regularity, the number of model evaluations per MCMC step can be greatly reduced without biasing the Monte Carlo average. Numerical experiments demonstrate multiple order-of-magnitude reductions in the number of forward model evaluations used in representative ODE and PDE inference problems, with both synthetic and real data.

show abstract

Regular Perturbation of V-Geometrically Ergodic Markov Chains

Cited by 4 publications

References 15 publications

Convergence and Optimality of Policy Gradient Methods in Weakly Smooth Settings

Convergence and Optimality of Policy Gradient Methods in Weakly Smooth Settings

Robustness of iterated function systems of Lipschitz maps

Accelerating Asymptotically Exact MCMC for Computationally Intensive Models via Local Approximations

Contact Info

Product

Resources

About