An Integral Representation of the Relative Entropy

Hirata, Miku; Nemoto, Aya; Yoshida, Hiroaki

doi:10.3390/e14081469

Cited by 5 publications

(7 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An alternative proof of this identity by direct calculation with integrations by part has been given in [5]. It should be noted here that the reference measure does move by the same heat equation in the formula of Lemma 1.…”

Section: Introductionmentioning

confidence: 91%

See 1 more Smart Citation

A Dissipation of Relative Entropy by Diffusion Flows

Yoshida

2016

Entropy

Self Cite

View full text Add to dashboard Cite

Given a probability measure, we consider the diffusion flows of probability measures associated with the partial differential equation (PDE) of Fokker-Planck. Our flows of the probability measures are defined as the solution of the Fokker-Planck equation for the same strictly convex potential, which means that the flows have the same equilibrium. Then, we shall investigate the time derivative for the relative entropy in the case where the object and the reference measures are moving according to the above diffusion flows, from which we can obtain a certain dissipation formula and also an integral representation of the relative entropy.

show abstract

Section: Introductionmentioning

confidence: 91%

“…In this paper, we will treat the Focker-Planck equation with strictly convex potential as our continuity equation, which is because the first natural extension of the heat equation and the similar dissipation formula in Lemma 2 of Vérdu can be derived by the fundamental method, the integration by parts like in [5].…”

Section: Lemmamentioning

confidence: 99%

A Dissipation of Relative Entropy by Diffusion Flows

Yoshida

2016

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

“…As far as it was possible to determine, the first definition of the relative Fisher information was given by Otto and Villani [36], who defined it for the translationally-invariant case. Furthermore, this expression has been rediscovered or simply used in many applications thereafter in different problems and fields [22,[37][38][39][40][41][42][43][44]. Furthermore, it seems that the first general analysis of the relative Fisher information was presented by the author in [45].…”

Section: Relative Fisher Information Type Imentioning

confidence: 99%

“…Stam's inequality [8,9,40,[47][48][49][50] states a lower bound for Fisher information, which links Fisher information and Shannon's entropy power. However, this expression is limited to the special case where the parameters in the Fisher information expression correspond to a location parameter.…”

Section: Lower Bound For Fisher Informationmentioning

confidence: 99%

Fisher Information Properties

Zegers

2015

Entropy

View full text Add to dashboard Cite

A set of Fisher information properties are presented in order to draw a parallel with similar properties of Shannon differential entropy. Already known properties are presented together with new ones, which include: (i) a generalization of mutual information for Fisher information; (ii) a new proof that Fisher information increases under conditioning; (iii) showing that Fisher information decreases in Markov chains; and (iv) bound estimation error using Fisher information. This last result is especially important, because it completes Fano's inequality, i.e., a lower bound for estimation error, showing that Fisher information can be used to define an upper bound for this error. In this way, it is shown that Shannon's differential entropy, which quantifies the behavior of the random variable, and the Fisher information, which quantifies the internal structure of the density function that defines the random variable, can be used to characterize the estimation error.

show abstract

“…Recently, the RFI has been the subject of intense investigations in a quest to obtain a better perspective of its physical implications [10][11][12][13][14], and to formally establish its role in information theory and estimation theory [15,16]. Still, many of the fundamental properties and physical implications of the RFI remain uninvestigated.…”

Section: Introductionmentioning

confidence: 99%

Legendre transform structure and extremal properties of the relative Fisher information

Venkatesan¹,

Plastino²

2014

Physics Letters A

View full text Add to dashboard Cite

Variational extremization of the relative Fisher information (RFI, hereafter) is performed. Reciprocity relations, akin to those of thermodynamics are derived, employing the extremal results of the RFI expressed in terms of probability amplitudes. A time independent Schrödinger-like equation (Schrödinger-like link) for the RFI is derived. The concomitant Legendre transform structure (LTS, hereafter) is developed by utilizing a generalized RFI-Euler theorem, which shows that the entire mathematical structure of thermodynamics translates into the RFI framework, both for equilibrium and non-equilibrium cases. The qualitatively distinct nature of the present results vis-á-vis those of prior studies utilizing the Shannon entropy and/or the Fisher information measure (FIM, hereafter) is discussed. A principled relationship between the RFI and the FIM frameworks is derived. The utility of this relationship is demonstrated by an example wherein the energy eigenvalues of the Schrödinger-like link for the RFI is inferred solely using the quantum mechanical virial theorem and the LTS of the RFI.

show abstract

An Integral Representation of the Relative Entropy

Cited by 5 publications

References 18 publications

A Dissipation of Relative Entropy by Diffusion Flows

A Dissipation of Relative Entropy by Diffusion Flows

Fisher Information Properties

Legendre transform structure and extremal properties of the relative Fisher information

Contact Info

Product

Resources

About