The Bregman Chord Divergence

Abstract

Distances are fundamental primitives whose choice significantly impacts the performances of algorithms in machine learning and signal processing. However selecting the most appropriate distance for a given task is an endeavor. Instead of testing one by one the entries of an ever-expanding dictionary of ad hoc distances, one rather prefers to consider parametric classes of distances that are exhaustively characterized by axioms derived from first principles. Bregman divergences are such a class. However fine-tuning a Bregman divergence is delicate since it requires to smoothly adjust a functional generator. In this work, we propose an extension of Bregman divergences called the Bregman chord divergences. This new class of distances does not require gradient calculations, uses two scalar parameters that can be easily tailored in applications, and generalizes asymptotically Bregman divergences.

An Elementary Introduction to Information Geometry

“…Thus we have: or equivalently the following inequalities: That is, we recover for (i.e., and ) the -skewed Jensen divergence [ 102 ]: Now, a consequence of the chordal slope lemma is that for a strictly convex and differentiable function F , we have: This can be geometrically visualized in Figure A1 . That is, We recognize the expressions of the Bregman divergences [ 91 ]: and get: For multivariate strictly convex functions F , we observe that we can build a multivariate Bregman divergence from a family of 1D Bregman divergences [ 161 ] induced by the 1D strictly convex functions : Theorem A1. A multivariate Bregman divergence between two parameters and can be expressed as a univariate Bregman divergence for the generator induced by the parameters: where Proof.…”

“…For multivariate strictly convex functions F , we observe that we can build a multivariate Bregman divergence from a family of 1D Bregman divergences [ 161 ] induced by the 1D strictly convex functions :…”