On $f$-divergences between Cauchy distributions

Nielsen, Frank; Okamura, Kazuki

doi:10.48550/arxiv.2101.12459

Cited by 6 publications

(16 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In general, a sufficient condition for expressing the f -divergence as a series of order-k chi divergences is given in [21]. However, the condition…”

Section: Some Illustrating Examplesmentioning

confidence: 99%

See 1 more Smart Citation

A note on the $f$-divergences between multivariate location-scale families with either prescribed scale matrices or location parameters

Nielsen¹,

Okamura²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

We extend the result of Ali and Silvey [Journal of the Royal Statistical Society: Series B, 28.1 (1966), 131-142] who first reported that any f -divergence between two isotropic multivariate Gaussian distributions amounts to a corresponding strictly increasing scalar function of their corresponding Mahalanobis distance. We report sufficient conditions on the standard probability density function generating a multivariate location-scale family and the generator f in order to generalize this result. In that case, one can compare exactly f -divergences between densities of these location families via their Mahalanobis distances. In particular, this proves useful when the f -divergences are not available in closed-form as it is the case for example for the Jensen-Shannon divergence between multivariate isotropic Gaussian distributions. Furthermore, we show that the f -divergences between these multivariate location-scale families amount equivalently to f -divergences between corresponding univariate location-scale families. We present several applications of these results.

show abstract

“…In general, a sufficient condition for expressing the f -divergence as a series of order-k chi divergences is given in [21]. However, the condition…”

Section: Some Illustrating Examplesmentioning

confidence: 99%

“…However, even for asymmetric f -divergences like the Kullback-Leibler divergence, some parametric families of distributions yield symmetric divergences. This is the case for the f -divergences between isotropic Gaussian distributions or f -divergences between Cauchy distributions which are always symmetric [21].…”

Section: Introductionmentioning

confidence: 99%

A note on the $f$-divergences between multivariate location-scale families with either prescribed scale matrices or location parameters

Nielsen¹,

Okamura²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In [14], the following closed-form formula (using complex analysis in C) was proven for the Kullback-Leibler divergence between a Cauchy density and a mixture of two Cauchy densities:…”

Section: Information Geometry Of the Mixture Family Of Two Cauchy Dis...mentioning

confidence: 99%

“…In [14], the following closed-form formula was reported for the Jensen-Shannon divergence when θ = 1 2 :…”

Section: Information Geometry Of the Mixture Family Of Two Cauchy Dis...mentioning

confidence: 99%

“…Proposition 3 The Kullback-Leibler divergence, Jeffreys divergence and Jensen-Shannon divergence can be calculated in closed form formula for any two mixtures of Cauchy distributions with prescribed two components. Notice that closed-form formula for the Kullback-Leibler divergence and the Jensen-Shannon divergence between Cauchy distributions were reported in [14].…”

Section: Information Geometry Of the Mixture Family Of Two Cauchy Dis...mentioning

confidence: 99%

See 1 more Smart Citation

The analytic dually flat space of the mixture family of two prescribed distinct Cauchy distributions

Nielsen¹

2021

Preprint

Self Cite

View full text Add to dashboard Cite

In information geometry, a strictly convex and smooth function induces a dually flat Hessian manifold equipped with a pair of dual Bregman divergences, hereby termed a Bregman manifold. Two common types of such Bregman manifolds met in statistics are (1) the exponential family manifolds induced by the cumulant functions of regular exponential families, and (2) the mixture family manifolds induced by the Shannon negentropies of statistical mixture families with prescribed linearly independent mixture components. However, the differential entropy of a mixture of continuous probability densities sharing the same support is hitherto not known in closed form making implementation of mixture family manifolds in practice difficult. In this work, we report an exception: The family of mixtures of two prescribed and distinct Cauchy distributions. We exemplify the explicit construction of a dually flat manifold induced by the differential negentropy for this very particular setting. This construction allows one to use the geometric toolbox of Bregman algorithms, and to obtain closed-form formula (albeit being large) for the Kullback-Leibler divergence and the Jensen-Shannon divergence between two mixtures of two prescribed Cauchy components.

show abstract

Information geometry, Pythagorean-theorem extension, and Euclidean distance behind optical sensing via spectral analysis

Желтиков¹

2022

Laser Phys. Lett.

View full text Add to dashboard Cite

We present an information-geometric perspective on a generic spectral-analysis task pertaining to a vast class of optical measurements in which a parameter θ needs to be evaluated from θ-dependent spectral features in a measurable optical readout. We show that the spectral shift and line broadening driven by small Δθ variations can be isolated as orthogonal components in a Pythagorean-theorem extension for a Euclidean distance in the space of probability distributions, representing the Δθ-induced information gain, expressible via the relative entropy and the pertinent Fisher information. This result offers important insights into the limits of optical signal analysis, as well as into the ultimate spectral resolution and the limiting sensitivity of a vast class of optical measurements. As one example, we derive a physically transparent closed-form analytical solution for the information-theory bound on the precision of all-optical temperature sensors based on color centers in diamond.

show abstract

On $f$-divergences between Cauchy distributions

Cited by 6 publications

References 48 publications

A note on the $f$-divergences between multivariate location-scale families with either prescribed scale matrices or location parameters

A note on the $f$-divergences between multivariate location-scale families with either prescribed scale matrices or location parameters

The analytic dually flat space of the mixture family of two prescribed distinct Cauchy distributions

Information geometry, Pythagorean-theorem extension, and Euclidean distance behind optical sensing via spectral analysis

Contact Info

Product

Resources

About