Exponential ergodicity for Markov processes with random switching

Cloez, Bertrand; Hairer, Martin

doi:10.3150/13-bej577

Cited by 133 publications

(159 citation statements)

References 34 publications

Supporting

Mentioning

159

Contrasting

Order By: Relevance

“…Their convergence has been widely studied in the litterature, and we can differentiate several approaches. For instance, there are so-called "Meyn-and-Tweedie" methods (or Foster-Lyapunov criteria, see [MT93,HM11,HMS11,CH15]) which provide qualitative convergence under mild conditions; we can follow this approach to provide qualitative properties for our inhomogeneous Markov chain. However, the speed is usually not explicit or very poor.…”

Section: Introductionmentioning

confidence: 99%

Ergodicity of inhomogeneous Markov chains through asymptotic pseudotrajectories

Benaı̈m¹,

Bouguet²,

Cloez³

2017

Ann. Appl. Probab.

Self Cite

View full text Add to dashboard Cite

In this work, we consider an inhomogeneous (discrete time) Markov chain and are interested in its long time behavior. We provide sufficient conditions to ensure that some of its asymptotic properties can be related to the ones of a homogeneous (continuous time) Markov process. Renowned examples such as a bandit algorithms, weighted random walks or decreasing step Euler schemes are included in our framework. Our results are related to functional limit theorems, but the approach differs from the standard "Tightness/Identification" argument; our method is unified and based on the notion of pseudotrajectories on the space of probability measures.

show abstract

Section: Introductionmentioning

confidence: 99%

Ergodicity of inhomogeneous Markov chains through asymptotic pseudotrajectories

Benaı̈m¹,

Bouguet²,

Cloez³

2017

Ann. Appl. Probab.

Self Cite

View full text Add to dashboard Cite

show abstract

“…[7], [5], [4], and [1] all study invariant measures for such processes. Our work shows that the existence of such invariant measures may depend in a complicated way on the switching rates.…”

Section: Introductionmentioning

confidence: 99%

“…[7], [5], [4], and [1] all study invariant measures for such processes. Our work shows that the existence of…”

Section: Introductionmentioning

confidence: 99%

Sensitivity to switching rates in stochastically switched odes

Lawley

Mattingly

Reed

2014

Communications in Mathematical Sciences

View full text Add to dashboard Cite

Abstract. We consider a stochastic process driven by a linear ordinary differential equation whose right-hand side switches at exponential times between a collection of different matrices. We construct planar examples that switch between two matrices where the individual matrices and the average of the two matrices are all Hurwitz (all eigenvalues have strictly negative real part), but nonetheless the process goes to infinity at large time for certain values of the switching rate. We further construct examples in higher dimensions where again the two individual matrices and their averages are all Hurwitz, but the process has arbitrarily many transitions between going to zero and going to infinity at large time as the switching rate varies. In order to construct these examples, we first prove in general that if each of the individual matrices is Hurwitz, then the process goes to zero at large time for sufficiently slow switching rate and if the average matrix is Hurwitz, then the process goes to zero at large time for sufficiently fast switching rate. We also give simple conditions that ensure the process goes to zero at large time for all switching rates.Key words. Ergodicity, piecewise deterministic Markov process, switched dynamical systems, hybrid switching system, planar switched systems, linear differential equations.AMS subject classifications. 60J75, 93E15, 37H15, 34F05, 34D23. IntroductionWe consider the stochastic process (X t ) t≥0 ∈ R d where X t solvesẊ t = A It X t with I t a Markov process on a finite set E and {A i } i∈E a set of d × d real matrices. The stability of this system when the switching process I t is deterministic has been extensively studied in the past decade; see [2] and [12].In [6], the authors study the stochastic problem in the plane with I t a Markov process and E = {0,1}. The authors assume both A 0 and A 1 are Hurwitz (all eigenvalues have strictly negative real part) and prove the surprising result that ||X t || may converge to 0 or +∞ as t → ∞ depending on the switching rate as long as an average matrixĀ = λA 0 + (1 − λ)A 1 has a positive eigenvalue for some λ ∈ (0,1).In this paper, we show that the assumption that the average matrix has a positive eigenvalue is not necessary to ensure a blowup. Specifically, we construct examples in the plane where A 0 , A 1 , andĀ = λA 0 + (1 − λ)A 1 are all Hurwitz, but ||X t || → +∞ almost surely as t → ∞ for certain values of the switching rate. This is significant for the general study of switching processes because it shows that the dynamics of the switching process can be very different from both the individual dynamics (in this case, the A i 's) and the averaged dynamics (in this case,Ā). These planar examples are also interesting because they have multiple transitions between ||X t || going to 0 and going to +∞ at large time as the switching rate varies. Furthermore, we construct examples in higher dimensions that have arbitrarily many such phase transitions.Recently researchers have devoted considerable attention to randomly switc...

show abstract

“…In the last decade, a series of works have been devoted for the questions above [1,[3][4][5]7,11,39,46]. In the simplest setting, the transition rates are constants, λ(x, y, y ) = λ(y, y ), and X t is driven by the linear equation (1.2).…”

Section: Introductionmentioning

confidence: 99%

Moment bounds and geometric ergodicity of diffusions with random switching and unbounded transition rates

Tong

Majda

2016

Res Math Sci

View full text Add to dashboard Cite

A diffusion with random switching is a Markov process that consists of a stochastic differential equation part X t and a continuous Markov jump process part Y t . Such systems have a wide range of applications, where the transition rates of Y t may not be bounded or Lipschitz. A new analytical framework is developed to understand the stability and ergodicity of these processes and allows for genuinely unbounded transition rates. Assuming the averaged dynamics is dissipative, the first part of this paper explicitly demonstrates how to construct a polynomial Lyapunov function and furthermore moment bounds. When the transition rates have multiple scales, this construction comes interestingly as a dual process of the averaging of fast transitions. The coefficients of the Lyapunov function can be seen as the potential dissipation of each regime in different scales, and a comparison principle comes naturally under this interpretation. On the basis of these results, the second part of this paper establishes geometric ergodicity for the joint processes. This can be achieved in two scenarios. If there is a commonly accessible regime that satisfies the minorization condition, the geometric convergence to the ergodic measure takes place in the total variation distance. If there is contraction on average, the geometric convergence takes place in a proper Wasserstein distance and is proved through an application of the asymptotic coupling framework. BackgroundDiffusions with random switching are stochastic processes consisting of two components: a diffusion process X t in R d and a continuous jump process Y t on a finite set F . The dynamics of X t follows a stochastic differential equation (SDE)(1.1) Throughout, we assume b(x, y) is C 1+δ and σ (x, y) is C 2+δ in x for some δ > 0. The behavior of Y t can be described by a transition rate function λ (x, y,ỹ), in other wordsWe denote the total transition rate asλ(x, y) = ỹ =y λ (x, y,ỹ), and the joint process as Z t = (X t , Y t ), which takes place in the space E = R d × F .

show abstract

Exponential ergodicity for Markov processes with random switching

Cited by 133 publications

References 34 publications

Ergodicity of inhomogeneous Markov chains through asymptotic pseudotrajectories

Ergodicity of inhomogeneous Markov chains through asymptotic pseudotrajectories

Sensitivity to switching rates in stochastically switched odes

Moment bounds and geometric ergodicity of diffusions with random switching and unbounded transition rates

Contact Info

Product

Resources

About