Intrinsic Losses Based on Information Geometry and Their Applications

Rong, Yao; Tang, Mengjiao; Zhou, Jie

doi:10.3390/e19080405

Cited by 6 publications

(4 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This type of loss functions can be obtained directly from the measure of discrepancies between two probability density functions. In literature, the term 'intrinsic loss' is used alternatively to indicate this kind of loss functions (Rong, Tang, and Zhou 2017;Ni and Sun 2021).…”

Section: Kullback-leibler Distance Loss Functionmentioning

confidence: 99%

A Relationship between Classical and Bayesian Estimation Procedures through Fisher Information

Bhunia,

Seal

2024

AJS

View full text Add to dashboard Cite

The aim of this article is to investigate the association between the classical and Bayesian approaches through Fisher information. For any particular distribution, the computation of Fisher information is quite significant, as it provides the amount of information about the unknown parameter inferred from the observed data and is related to classical methods of estimation. Also, in the light of some prior knowledge, we may estimate the unknown parameter through Bayesian approach. Specifically, we want to see a relationship between information and Bayes estimation. In this article, the scale parameter of the one-parameter exponential distribution is estimated under the weighted squared error and Kullback-Leibler distance loss functions. The information acquired from both the classical and Bayesian methodologies have been connected through the risk intensity and error intensity which have been introduced in this article. The results of extensive simulation studies using these intensity measures show that the Bayes estimator performs more intensely as the amount of Fisher information increases. It is seen that the Fisher information, which is pivotal to many classical estimation methods, has a relationship with the Bayesian method depending on prior distribution, at least in this case, as the intensity measures of the Bayes estimator decrease with the increase in information. Further, to comprehend the theoretical notion of association, two real-life datasets have been included to show usefulness in practical field.

show abstract

Section: Kullback-leibler Distance Loss Functionmentioning

confidence: 99%

A Relationship between Classical and Bayesian Estimation Procedures through Fisher Information

Bhunia,

Seal

2024

AJS

View full text Add to dashboard Cite

show abstract

“…If g(x) is a reversible function, the PDF of x and y satisfy: p y (y; θ) dg(x) dx = p x (x; θ). (7) According to the Fisher-Neyman factorization theorem [27], y is the sufficient statistic of x, so G S (θ) = G S (θ).…”

Section: Corollary 2 If G(x) Is a Reversible Function G S (θ) = G S (θ)mentioning

confidence: 99%

“…Given the Fisher information matrix as the Riemannian metric, the distance between any two points (probability distributions) can be calculated [6]. In such a manifold, the distance between two points stands for the intrinsic measure for the dissimilarity between two probability distributions [7]. As information geometry provides a new perspective on signal processing, there are many applications of it.…”

Section: Introductionmentioning

confidence: 99%

Isometric Signal Processing under Information Geometric Framework

Hao

Cheng

Wang

2019

Entropy

View full text Add to dashboard Cite

Information geometry is the study of the intrinsic geometric properties of manifolds consisting of a probability distribution and provides a deeper understanding of statistical inference. Based on this discipline, this letter reports on the influence of the signal processing on the geometric structure of the statistical manifold in terms of estimation issues. This letter defines the intrinsic parameter submanifold, which reflects the essential geometric characteristics of the estimation issues. Moreover, the intrinsic parameter submanifold is proven to be a tighter one after signal processing. In addition, the necessary and sufficient condition of invariant signal processing of the geometric structure, i.e., isometric signal processing, is given. Specifically, considering the processing with the linear form, the construction method of linear isometric signal processing is proposed, and its properties are presented in this letter.

show abstract

“…The Wasserstein distance as a geodesic distance on the Riemannian manifold equipped with the Wasserstein metric is usually used for optimal transportation [ 21 , 22 ], while the Fisher metric applies primarily to information science. Therefore, by endowing such space with a natural Riemannian structure (i.e., the Fisher metric and Levi–Civita connection), the resulting geodesic distance, also called the Rao distance, has been taken as an intrinsic measure for the dissimilarity between two PDFs and then applied in wide fields such as neural networks [ 23 , 24 ], signal processing [ 25 , 26 ], and statistical inference [ 27 , 28 ].…”

Section: Introductionmentioning

confidence: 99%

Multisensor Estimation Fusion on Statistical Manifold

Chen

Zhou

2022

Entropy

Self Cite

View full text Add to dashboard Cite

In the paper, we characterize local estimates from multiple distributed sensors as posterior probability densities, which are assumed to belong to a common parametric family. Adopting the information-geometric viewpoint, we consider such family as a Riemannian manifold endowed with the Fisher metric, and then formulate the fused density as an informative barycenter through minimizing the sum of its geodesic distances to all local posterior densities. Under the assumption of multivariate elliptical distribution (MED), two fusion methods are developed by using the minimal Manhattan distance instead of the geodesic distance on the manifold of MEDs, which both have the same mean estimation fusion, but different covariance estimation fusions. One obtains the fused covariance estimate by a robust fixed point iterative algorithm with theoretical convergence, and the other provides an explicit expression for the fused covariance estimate. At different heavy-tailed levels, the fusion results of two local estimates for a static target display that the two methods achieve a better approximate of the informative barycenter than some existing fusion methods. An application to distributed estimation fusion for dynamic systems with heavy-tailed process and observation noises is provided to demonstrate the performance of the two proposed fusion algorithms.

show abstract

Intrinsic Losses Based on Information Geometry and Their Applications

Cited by 6 publications

References 41 publications

A Relationship between Classical and Bayesian Estimation Procedures through Fisher Information

A Relationship between Classical and Bayesian Estimation Procedures through Fisher Information

Isometric Signal Processing under Information Geometric Framework

Multisensor Estimation Fusion on Statistical Manifold

Contact Info

Product

Resources

About