Log-Determinant Divergences Revisited: Alpha-Beta and Gamma Log-Det Divergences

Abstract

This work reviews and extends a family of log-determinant (log-det) divergences for symmetric positive definite (SPD) matrices and discusses their fundamental properties. We show how to use parameterized Alpha-Beta (AB) and Gamma log-det divergences to generate many well-known divergences; in particular, we consider the Stein's loss, the S-divergence, also called Jensen-Bregman LogDet (JBLD) divergence, Logdet Zero (Bhattacharyya) divergence, Affine Invariant Riemannian Metric (AIRM), and other divergences. Moreover, we establish links and correspondences between log-det divergences and visualise them on an alpha-beta plane for various sets of parameters. We use this unifying framework to interpret and extend existing similarity measures for semidefinite covariance matrices in finite-dimensional Reproducing Kernel Hilbert Spaces (RKHS). This paper also shows how the Alpha-Beta family of log-det divergences relates to the divergences of multivariate and multilinear normal distributions. Closed form formulas are derived for Gamma divergences of two multivariate Gaussian densities; the special cases of the Kullback-Leibler, Bhattacharyya, Rényi, and Cauchy-Schwartz divergences are discussed. Symmetrized versions of log-det divergences are also considered and briefly reviewed. Finally, a class of divergences is extended to multiway divergences for separable covariance (or precision) matrices.

Optimization of Alpha-Beta Log-Det Divergences and their Application in the Spatial Filtering of Two Class Motor Imagery Movements

“…As it can be observed in Figure 1 the AB log-det divergence generalizes several existing log-det matrix divergences, like: the Stein's loss, the S-divergence, the Alpha and Beta log-det families of divergences and the geodesic distance between covariance matrices (the squared Riemannian metric), among others (see Table 1 in [6] for a comprehensive list). …”

“…Indeed, this is usual case in the comparison of probability density functions and also of their associated covariance matrices. The present contribution may be seen as a continuation of the work in [6], where we defined the Alpha-Beta Log-Det family of divergences between Symmetric and Positive Definite (SPD) matrices and studied its properties. The Alpha-Beta Log-Det family unifies under the same framework many existing Log-Det divergences and connects them smoothly, through intermediate…”

“…In this paper, we propose an extension of the existing KL to the criterion of the Alpha-Beta log-det divergence (AB log-det) between the class-conditional covariances defined as [6] …”

“…Thus, the spatial filter matrix W ∈ R n×p is used to reduce the dimensionality of the samples from n to p with the linear compression transformation y = W T x ∈ R p . It is shown in [6] that, after applying this compression to the arguments of the divergence, the resulting output covariance matrices W T PW and W T QW are more similar than in the original space, as shown in the below equation…”

Testing for Differences in Path Forecast Accuracy: Forecast-Error Dynamics Matter

“…A common class of scoring rules approximate the normal density and only depend on the first and second moments; see Gneiting and Raftery (2007). These scoring rules are also closely related to a wider class of loss functions; see Cichocki et al (2015). The next section constructs a likelihood ratio test of differences in path forecast accuracy using the joint density as a loss function.…”