DreamNet: A Deep Riemannian Manifold Network for SPD Matrix Learning

Abstract

Symmetric positive definite (SPD) matrix has been demonstrated to be an effective feature descriptor in many scientific areas, as it can encode spatiotemporal statistics of the data adequately on a curved Riemannian manifold, i.e., SPD manifold. Although there are many different ways to design network architectures for SPD matrix nonlinear learning, very few solutions explicitly mine the geometrical dependencies of features at different layers. Motivated by the great success of self-attention mechanism in capturing long-range relationships, an SPD manifold self-attention mechanism (SMSA) is proposed in this paper using some manifold-valued geometric operations, mainly the Riemannian metric, Riemannian mean, and Riemannian optimization. Then, an SMSA-based geometric learning module (SMSA-GLM) is designed for the sake of improving the discrimination of the generated deep structured representations. Extensive experimental results achieved on three benchmarking datasets show that our modification against the baseline network further alleviates the information degradation problem and leads to improved accuracy.

Learning Submanifold-Specific Normalization and Attention with SPD Matrices for Visual Classification

“…The non-singular covariance matrices known as the Symmetric Positive Definite (SPD) matrices which can form the SPD Riemannian manifold. Classification tasks with SPD manifold for video data have received increasing attention in recent years, such as video-based facial emotion recognition [1][2][3][4],dynamic scene classification [5][6][7],action recognition [8][9][10][11][12], and video-based face recognition [13][14][15][16].…”

“…Inspired from U-Net [20], Wang et al redesigned the covariance matrix to create the U-SPDNet [12] network, allowing the feature extraction component to learn more informative low-dimensional mappings. DreamNet [11] addressed the problem of poor performance in deepening the SPDNet by using a stacked U-SPDNet approach and reconstructing SPD matrices.…”