Rate-Invariant Analysis of Covariance Trajectories

Abstract

Statistical classification of actions in videos is mostly performed by extracting relevant features, particularly covariance features, from image frames and studying time series associated with temporal evolutions of these features. A natural mathematical representation of activity videos is in form of parameterized trajectories on the covariance manifold, i.e. the set of symmetric, positive-definite matrices (SPDMs). The variable execution-rates of actions implies variable parameterizations of the resulting trajectories, and complicates their classification. Since action classes are invariant to execution rates, one requires rate-invariant metrics for comparing trajectories. A recent paper represented trajectories using their transported square-root vector fields (TSRVFs), defined by parallel translating scaled-velocity vectors of trajectories to a reference tangent space on the manifold. To avoid arbitrariness of selecting the reference and to reduce distortion introduced during this mapping, we develop a purely intrinsic approach where SPDM trajectories are represented by redefining their TSRVFs at the starting points of the trajectories, and analyzed as elements of a vector bundle on the manifold. Using a natural Riemannain metric on vector bundles of SPDMs, we compute geodesic paths and geodesic distances between trajectories in the quotient space of this vector bundle, with respect to the reparameterization group. This makes the resulting comparison of trajectories invariant to their re-parameterization. We demonstrate this framework on two applications involving video classification: visual speech recognition or lip-reading and hand-gesture recognition. In both cases we achieve results either comparable to or better than the current literature.

Analyzing Dynamical Brain Functional Connectivity as Trajectories on Space of Covariance Matrices

“…In order to quantify differences in FCs, represented by covariance matrices, we need a metric structure on the manifold of covariance matrices or SPDMs. While there are several Riemannian structures in the literature [11,13,14], we use the one introduced in [11], since it has the advantage of having close forms for many necessary operations we need on the SPDM manifold, e.g., geodesic distance, parallel transport, exponential map, inverse exponential map. Zhang et al [11] also have demonstrated that this metric is superior over other metrics such as the log-Euclidean one [13] in analyzing dynamic FCs.…”

Is Affine-Invariance Well Defined on SPD Matrices? A Principled Continuum of Metrics

“…The affine-invariant metric [2,8,3] and the polar-affine metric [12] are different but they both provide a Riemannian symmetric structure to the manifold of SPD matrices. Moreover, both claim to be very naturally introduced.…”

“…This change of basis can prevent from numerical approximations of the differential but one must keep in mind that V = V ′ in general. This identification was already used for the polar-affine metric (f = pow 2 ) in [12] without explicitly mentioning.…”

“…It was actually shown that there exists not only one but a one-parameter family that is invariant under these affine transformations [9]. More recently, [10,11,12] introduced another Riemannian symmetric structure that does not belong to the previous one-parameter family: the polar-affine metric.…”

Discovering common change-point patterns in functional connectivity across subjects

“…While there are several Riemannian structures used in the literature (including one in Pennec et al (2006)), we briefly summarize the idea used in this paper. More details can be found in Su et al (2012) and Zhang et al (2015).…”