The Burbea-Rao and Bhattacharyya centroids

Abstract

Abstract-We study the centroid with respect to the class of information-theoretic Burbea-Rao divergences that generalize the celebrated Jensen-Shannon divergence by measuring the nonnegative Jensen difference induced by a strictly convex and differentiable function. Although those Burbea-Rao divergences are symmetric by construction, they are not metric since they fail to satisfy the triangle inequality. We first explain how a particular symmetrization of Bregman divergences called JensenBregman distances yields exactly those Burbea-Rao divergences. We then proceed by defining skew Burbea-Rao divergences, and show that skew Burbea-Rao divergences amount in limit cases to compute Bregman divergences. We then prove that Burbea-Rao centroids are unique, and can be arbitrarily finely approximated by a generic iterative concave-convex optimization algorithm with guaranteed convergence property. In the second part of the paper, we consider the Bhattacharyya distance that is commonly used to measure overlapping degree of probability distributions. We show that Bhattacharyya distances on members of the same statistical exponential family amount to calculate a Burbea-Rao divergence in disguise. Thus we get an efficient algorithm for computing the Bhattacharyya centroid of a set of parametric distributions belonging to the same exponential families, improving over former specialized methods found in the literature that were limited to univariate or "diagonal" multivariate Gaussians. To illustrate the performance of our Bhattacharyya/Burbea-Rao centroid algorithm, we present experimental performance results for k-means and hierarchical clustering methods of Gaussian mixture models.

On the chi square and higher-order chi distances for approximating f-divergences

“…of the same exponential family amount to compute a Bregman divergence on swapped natural parameters [9]: KL(X 1 :…”

Generalized Bhattacharyya and Chernoff upper bounds on Bayes error using quasi-arithmetic means

“…The power mean induced by f (x) = x 1 t (with t = − ν+d 2 ) tends to the geometric mean, and we get the well-known Bhattacharyya coefficient bound (for α = 1 2 ) on central multivariate Gaussians [16] (see Eq. 35):…”

“…Wlog., let the class-conditional probabilities p 1 and p 2 belong to the same exponential family, then we have [11,16]:…”

“…Space Θ is the parameter domain, called the natural parameter space [16]. F is a strictly convex and differentiable convex function called the log-normalizer.…”

“…distributions are exponential families in disguise [11,16] for which the skewed affinity coefficient ρ α (i.e., similarity distance within [0, 1]) can be computed in closed-form. An exponential family is a family F of distributions:…”