Fisher Vector Coding for Covariance Matrix Descriptors Based on the Log-Euclidean and Affine Invariant Riemannian Metrics

Zhang

et al. 2022

Brain Science Advances

The brain–computer interface (BCI) technology has received lots of attention in the field of scientific research because it can help disabled people improve their quality of life. Steady‐state visual evoked potential (SSVEP) is the most researched BCI experimental paradigm, which offers the advantages of high signal‐to‐noise ratio and short training‐time requirement by users. In a complete BCI system, the two most critical components are the experimental paradigm and decoding algorithm. However, a systematic combination of the SSVEP experimental paradigm and decoding algorithms is missing in existing studies. In the present study, the transient visual evoked potential, SSVEP, and various improved SSVEP paradigms are compared and analyzed, and the problems and development bottlenecks in the experimental paradigm are finally pointed out. Subsequently, the canonical correlation analysis and various improved decoding algorithms are introduced, and the opportunities and challenges of the SSVEP decoding algorithm are discussed.

Section: Decoding Algorithm For Ssvepmentioning

confidence: 99%

Review of brain–computer interface based on steady‐state visual evoked potential

Zhang

et al. 2022

Brain Science Advances

“…This has the advantage of more accurately capturing the underlying scatter of the data points (that are covariance matrices) than is possible with methods that treat data points as elements in a vector space. For many applications, the log-Euclidean framework has shown competitive results compared to the affine invariant Riemannian one [31,32]. This log-Euclidean framework is considered in this paper for its efficiency and ease of use.…”

Section: Log-euclidean Framework For Second-order Statistics Of Cnn Featuresmentioning

confidence: 99%

Ensemble Learning Approaches Based on Covariance Pooling of CNN Features for High Resolution Remote Sensing Scene Classification

et al. 2020

Self Cite

Remote sensing image scene classification, which consists of labeling remote sensing images with a set of categories based on their content, has received remarkable attention for many applications such as land use mapping. Standard approaches are based on the multi-layer representation of first-order convolutional neural network (CNN) features. However, second-order CNNs have recently been shown to outperform traditional first-order CNNs for many computer vision tasks. Hence, the aim of this paper is to show the use of second-order statistics of CNN features for remote sensing scene classification. This takes the form of covariance matrices computed locally or globally on the output of a CNN. However, these datapoints do not lie in an Euclidean space but a Riemannian manifold. To manipulate them, Euclidean tools are not adapted. Other metrics should be considered such as the log-Euclidean one. This consists of projecting the set of covariance matrices on a tangent space defined at a reference point. In this tangent plane, which is a vector space, conventional machine learning algorithms can be considered, such as the Fisher vector encoding or SVM classifier. Based on this log-Euclidean framework, we propose a novel transfer learning approach composed of two hybrid architectures based on covariance pooling of CNN features, the first is local and the second is global. They rely on the extraction of features from models pre-trained on the ImageNet dataset processed with some machine learning algorithms. The first hybrid architecture consists of an ensemble learning approach with the log-Euclidean Fisher vector encoding of region covariance matrices computed locally on the first layers of a CNN. The second one concerns an ensemble learning approach based on the covariance pooling of CNN features extracted globally from the deepest layers. These two ensemble learning approaches are then combined together based on the strategy of the most diverse ensembles. For validation and comparison purposes, the proposed approach is tested on various challenging remote sensing datasets. Experimental results exhibit a significant gain of approximately 2% in overall accuracy for the proposed approach compared to a similar state-of-the-art method based on covariance pooling of CNN features (on the UC Merced dataset).

2018 Eighth International Conference on Image Processing Theory, Tools and Applications (IPTA)

“…Recently, we have proposed to extend the FV descriptors to SPD features. This has involved the Log-Euclidean Fisher vectors (LE FV) and the Riemannian Fisher vectors (RFV) [26], [27], [28]. Although the log-Euclidean metric do not yield full affine invariance compared to the affine invariant Riemannian metric, it is invariant by similarity (orthogonal transformation and scaling).…”

Section: Introductionmentioning

confidence: 99%

Image classification based on log-Euclidean Fisher Vectors for covariance matrix descriptors

Akodad

Bombrun

Yaacoub

et al. 2018

Self Cite

This paper introduces an image classification method based on the encoding of a set of covariance matrices. This encoding relies on Fisher vectors adapted to the log-Euclidean metric: the log-Euclidean Fisher vectors (LE FV). This approach is next extended to full local Gaussian descriptors composed by a set of local mean vectors and local covariance matrices. For that, the local Gaussian model is transformed to a zero-mean Gaussian model with an augmented covariance matrix. All these approaches are used to encode handcrafted or deep learning features. Finally, they are applied in a remote sensing application on the UC Merced dataset which consists in classifying land cover images. A sensitivity analysis is carried out to evaluate the potential of the proposed LE FV.