On information gain, Kullback-Leibler divergence, entropy production and the involution kernel

Lopes, Artur O.; Mengue, Jairo K.

doi:10.3934/dcds.2022026

Cited by 8 publications

(10 citation statements)

References 37 publications

(107 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Here we are interested in the Kullback-Leibler divergence (KL-divergence for short) of shift invariant probabilities (see (17) in [37])…”

Section: Kl-divergence and Dynamical Information Projectionsmentioning

confidence: 99%

“…From a Bayesian point of view, the probability µ 1 describes the prior probability and µ λ plays the role of the posterior probability in the inductive inference problem described by expression D KL (µ λ , µ 1 ) (see Section 2.10 in [10], [37] and [20]). The function log J λ − log J 1 should be considered as the likelihood function (see [23]).…”

Section: Kl-divergence and Dynamical Information Projectionsmentioning

confidence: 99%

“…A detailed study of the KL-divergence for equilibrium probabilities is described in [36], [37], [16], [17] and [18].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Geodesics and dynamical information projections on the manifold of Hölder equilibrium probabilities

Lopes¹,

Ruggiero²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

We consider here the discrete time dynamics described by a transformation T : M → M , where T is either the action of shift T = σ on the symbolic space M = {1, 2, ..., d} N , or, T describes the action of a d to 1 expanding transformation T : S 1 → S 1 of class C 1+α ( for example x → T (x) = d x (mod 1) ), where M = S 1 is the unit circle. It is known that the infinite-dimensional manifold N of equilibrium probabilities for Hölder potentials A : M → R is an analytical manifold and carries a natural Riemannian metric associated with the asymptotic variance. We show here that under the assumption of the existence of a Fourier-like Hilbert basis for the kernel of the Ruelle operator there exists geodesics paths. When T = σ and M = {0, 1} N such basis exists.In a different direction, we also consider the KL-divergence D KL (µ 1 , µ 2 ) for a pair of equilibrium probabilities. If D KL (µ 1 , µ 2 ) = 0, then µ 1 = µ 2 . Although D KL is not a metric in N , it describes the proximity between µ 1 and µ 2 . A natural problem is: for a fixed probability µ 1 ∈ N consider the probability µ 2 in a convex set of probabilities in N which minimizes D KL (µ 1 , µ 2 ). This minimization problem is a dynamical version of the main issues considered in information projections. We consider this problem in N , a case where all probabilities are dynamically invariant, getting explicit equations for the solution sought. Triangle and Pythagorean inequalities will be investigated.

show abstract

“…Here we are interested in the Kullback-Leibler divergence (KL-divergence for short) of shift invariant probabilities (see (17) in [37])…”

Section: Kl-divergence and Dynamical Information Projectionsmentioning

confidence: 99%

Section: Kl-divergence and Dynamical Information Projectionsmentioning

confidence: 99%

See 1 more Smart Citation

Geodesics and dynamical information projections on the manifold of Hölder equilibrium probabilities

Lopes¹,

Ruggiero²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The relative entropy is also known as the Kullback-Leibler divergence. For proofs of general results on the topic in the case of the shift we refer the reader to [8] for the case the alphabet is finite and for [28] in the case the alphabet is a compact metric space. We refer the reader to [21] for an application of Kullback-Leibler divergence in statistics.…”

Section: Preliminariesmentioning

confidence: 99%

Bayes posterior convergence for loss functions via almost additive Thermodynamic Formalism

Lopes¹,

Lopes²,

Varandas³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Statistical inference can be seen as information processing involving input information and output information that updates belief about some unknown parameters. We consider the Bayesian framework for making inferences about dynamical systems from ergodic observations, where the Bayesian procedure is based on the Gibbs posterior inference, a decision process generalization of standard Bayesian inference (see [3,22]) where the likelihood is replaced by the exponential of a loss function. In the case of direct observation and almost-additive loss functions, we prove an exponential convergence of the a posteriori measures a limit measure. Our estimates on the Bayes posterior convergence for direct observation are related but complementary to those in [30]. Our approach makes use of non-additive thermodynamic formalism and large deviation properties [24,25,38] instead of joinings.

show abstract

“…In [1], using properties of the involution kernel the authors proved Large Deviation Theorems for the zero-temperature limit in the case there exist selection ( [33] considers the case where it is not assumed selection). In another direction, in [24] it is shown that the involution kernel appears as a natural tool for the investigation of entropy production of µ A . If the potential A is symmetric then the entropy production of µ A is zero (see Section 7 in [24]).…”

Section: Introductionmentioning

confidence: 99%

The involution kernel and the dual potential for functions in the Walters family

Hataishi¹,

Lopes²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

First, we set a suitable notation. Points in {0, 1} Z−{0} = {0, 1} N × {0, 1} N = Ω − × Ω + , are denoted by (y|x) = (..., y 2 , y 1 |x 1 , x 2 , ...), where (x 1 , x 2 , ...) ∈ {0, 1} N , and (y 1 , y 2 , ...) ∈ {0, 1} N . The bijective map σ(..., y 2 , y 1 |x 1 , x 2 , ...) = (..., y 2 , y 1 , x 1 |x 2 , ...) is called the bilateral shift and acts on {0, 1} Z−{0} . Given A : {0, 1} N = Ω + → R we express A in the variable x, like A(x). In a similar way, given B : {0, 1} N = Ω − → R we express B in the variable y, like B(y). Finally, given W : Ω − × Ω + → R, we express W in the variable (y|x), like W (y|x). By abuse of notation we write A(y|x) = A(x) and B(y|x) = B(y). The probability µ A denotes the equilibrium probability for A : {0, 1} N → R.Given a continuous potential A : Ω + → R, we say that the continuous potential A * : Ω − → R is the dual potential of A, if there exists a continuous W : Ω − × Ω + → R, such that, for all (y|x) ∈ {0,We say that W is an involution kernel for A. The function W allows you to define an spectral projection in the linear space of the main eigenfunction of the Ruelle operator for A. Denote by θ : Ω − × Ω + → Ω − × Ω + the function θ(..., y 2 , y 1 |x 1 , x 2 , ...) = (..., x 2 , x 1 |y 1 , y 2 , ...). We say that A is symmetric if A * (θ(x|y)) = A(y|x) = A(x). To say that A is symmetric is equivalent to saying that µ A has zero entropy production. Given A, we describe explicit expressions for W and the dual potential A * , for A in a family of functions introduced by P. Walters. We present conditions for A to be symmetric and to be of twist type.

show abstract

On information gain, Kullback-Leibler divergence, entropy production and the involution kernel

Cited by 8 publications

References 37 publications

Geodesics and dynamical information projections on the manifold of Hölder equilibrium probabilities

Geodesics and dynamical information projections on the manifold of Hölder equilibrium probabilities

Bayes posterior convergence for loss functions via almost additive Thermodynamic Formalism

The involution kernel and the dual potential for functions in the Walters family

Contact Info

Product

Resources

About