A note on the $f$-divergences between multivariate location-scale families with either prescribed scale matrices or location parameters

Nielsen, Frank; Okamura, Kazuki

doi:10.48550/arxiv.2204.10952

Cited by 3 publications

(11 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We can express

where

(convex for

) and

. Then we rely on the proof of invariance of f -divergences under the action of the affine group (see Proposition 3 of [ 60 ] relying on a change of variable in the integral):

where I denotes the identity matrix. □…”

Section: Chernoff Information Between Gaussian Distributionsmentioning

confidence: 99%

“…Notice that it amounts to one eight of the squared Mahalanobis distance (see [ 60 ] for a detailed explanation).…”

Section: Chernoff Information Between Gaussian Distributionsmentioning

confidence: 99%

“…The matrix Burg distance is a matrix spectral distance [ 62 ]:

where the

’s are the eigenvalues of

. The reverse KLD divergence

is obtained by replacing

More generally, the f -divergences between centered Gaussian distributions are always matrix spectral divergences [ 60 ].…”

Section: Chernoff Information Between Gaussian Distributionsmentioning

confidence: 99%

See 2 more Smart Citations

Revisiting Chernoff Information with Likelihood Ratio Exponential Families

Nielsen

2022

Entropy

View full text Add to dashboard Cite

The Chernoff information between two probability measures is a statistical divergence measuring their deviation defined as their maximally skewed Bhattacharyya distance. Although the Chernoff information was originally introduced for bounding the Bayes error in statistical hypothesis testing, the divergence found many other applications due to its empirical robustness property found in applications ranging from information fusion to quantum information. From the viewpoint of information theory, the Chernoff information can also be interpreted as a minmax symmetrization of the Kullback–Leibler divergence. In this paper, we first revisit the Chernoff information between two densities of a measurable Lebesgue space by considering the exponential families induced by their geometric mixtures: The so-called likelihood ratio exponential families. Second, we show how to (i) solve exactly the Chernoff information between any two univariate Gaussian distributions or get a closed-form formula using symbolic computing, (ii) report a closed-form formula of the Chernoff information of centered Gaussians with scaled covariance matrices and (iii) use a fast numerical scheme to approximate the Chernoff information between any two multivariate Gaussian distributions.

show abstract

“…We can express

where

(convex for

) and

. Then we rely on the proof of invariance of f -divergences under the action of the affine group (see Proposition 3 of [ 60 ] relying on a change of variable in the integral):

where I denotes the identity matrix. □…”

Section: Chernoff Information Between Gaussian Distributionsmentioning

confidence: 99%

“…Notice that it amounts to one eight of the squared Mahalanobis distance (see [ 60 ] for a detailed explanation).…”

Section: Chernoff Information Between Gaussian Distributionsmentioning

confidence: 99%

“…The matrix Burg distance is a matrix spectral distance [ 62 ]:

where the

’s are the eigenvalues of

. The reverse KLD divergence

is obtained by replacing

More generally, the f -divergences between centered Gaussian distributions are always matrix spectral divergences [ 60 ].…”

Section: Chernoff Information Between Gaussian Distributionsmentioning

confidence: 99%

See 1 more Smart Citation

Revisiting Chernoff Information with Likelihood Ratio Exponential Families

Nielsen

2022

Entropy

View full text Add to dashboard Cite

show abstract

“…Notice that it amounts to one eight of the squared Mahalanobis distance (see [58] for a detailed explanation).…”

Section: Closed-form Formula For the Chernoff Information Between Uni...mentioning

confidence: 99%

“…More generally, the f -divergences between centered Gaussian distributions are always matrix spectral divergences [58].…”

Section: Fast Approximation Of the Chernoff Information Of Multivaria...mentioning

confidence: 99%

Revisiting Chernoff Information with Likelihood Ratio Exponential Families

Nielsen¹

2022

Preprint

Self Cite

View full text Add to dashboard Cite

The Chernoff information between two probability measures is a statistical divergence measuring their deviation defined as their maximally skewed Bhattacharyya distance. Although the Chernoff information was originally introduced for bounding the Bayes error in statistical hypothesis testing, the divergence found many other applications due to its empirical robustness property found in applications ranging from information fusion to quantum information. From the viewpoint of information theory, the Chernoff information can also be interpreted as a minmax symmetrization of the Kullback-Leibler divergence. In this paper, we first revisit the Chernoff information between two densities of a measurable Lebesgue space by considering the exponential families induced by their geometric mixtures: The so-called likelihood ratio exponential families. Second, we show how to (i) solve exactly the Chernoff information between any two univariate Gaussian distributions or get a closed-form formula using symbolic computing, (ii) report a closed-form formula of the Chernoff information of centered Gaussians with scaled covariance matrices and (iii) use a fast numerical scheme to approximate the Chernoff information between any two multivariate Gaussian distributions.

show abstract

A Simple Approximation Method for the Fisher–Rao Distance between Multivariate Normal Distributions

Nielsen

2023

Entropy

View full text Add to dashboard Cite

We present a simple method to approximate the Fisher–Rao distance between multivariate normal distributions based on discretizing curves joining normal distributions and approximating the Fisher–Rao distances between successive nearby normal distributions on the curves by the square roots of their Jeffreys divergences. We consider experimentally the linear interpolation curves in the ordinary, natural, and expectation parameterizations of the normal distributions, and compare these curves with a curve derived from the Calvo and Oller’s isometric embedding of the Fisher–Rao d-variate normal manifold into the cone of (d+1)×(d+1) symmetric positive–definite matrices. We report on our experiments and assess the quality of our approximation technique by comparing the numerical approximations with both lower and upper bounds. Finally, we present several information–geometric properties of Calvo and Oller’s isometric embedding.

show abstract

A note on the $f$-divergences between multivariate location-scale families with either prescribed scale matrices or location parameters

Cited by 3 publications

References 13 publications

Revisiting Chernoff Information with Likelihood Ratio Exponential Families

Revisiting Chernoff Information with Likelihood Ratio Exponential Families

Revisiting Chernoff Information with Likelihood Ratio Exponential Families

A Simple Approximation Method for the Fisher–Rao Distance between Multivariate Normal Distributions

Contact Info

Product

Resources

About