Efficient clustering on Riemannian manifolds: A kernelised random projection approach

Abstract

Reformulating computer vision problems over Riemannian manifolds has demonstrated superior performance in various computer vision applications. This is because visual data often forms a special structure lying on a lower dimensional space embedded in a higher dimensional space. However, since these manifolds belong to non-Euclidean topological spaces, exploiting their structures is computationally expensive, especially when one considers the clustering analysis of massive amounts of data. To this end, we propose an efficient framework to address the clustering problem on Riemannian manifolds. This framework implements random projections for manifold points via kernel space, which can preserve the geometric structure of the original space, but is computationally efficient. Here, we introduce three methods that follow our framework. We then validate our framework on several computer vision applications by comparing against popular clustering methods on Riemannian manifolds. Experimental results demonstrate that our framework maintains the performance of the clustering whilst massively reducing computational complexity by over two orders of magnitude in some cases.

Explicit discriminative representation for improved classification of manifold features

“…Despite its excellent performance, the current proposal requires to compute the kernel similarities for each query data to all training data, which could introduce a significant bottleneck in the process. Thus, in the future we would further develop the explicit feature map that only needs a significantly smaller set of training data by adapting techniques described in [3,41].…”

Data-independent Random Projections from the feature-map of the homogeneous polynomial kernel of degree two

“…In addition, it makes use of the specific data to be projected during the training stage (as opposed to standard RP, in which the projection matrix R is completely data-independent and no training stage is required). Later on, this kernelized version of RP was extended and applied to the problem of image clustering [32] . The authors proposed three versions of the algorithm: Kernelised Gaussian Random Projection (KG-RP), Kernelised Orthonormal Random Projection (KORP), and KPCA-based Random Projection (KPCA-RP).…”

“…Over the years, various authors [3,32] have explored the possibility of performing random projections from the feature space associated to different kernel functions. This is of interest because of two main reasons: (1) if the JL-lemma is satisfied, the pairwise distances between samples in the kernel feature space will be preserved in the resulting representation, so the low-dimensional projected points will preserve most of the structure from the kernel feature-space; and (2), since Random Projection preserves separability margins [3] , classification problems may become more lineally solvable after the Random Projection form a kernel feature space.…”

“…If this is the case, the classification accuracy of scalable linear classifiers will increase. Motivated by these advantages, Zhao et al recently proposed a kernelized variant of the original Random Projection Algorithm, capable of working with an arbitrary kernel function [32] . As a drawback, their proposal lost some of the beneficial features of the original Random Projection algorithm.…”

“…As a drawback, their proposal lost some of the beneficial features of the original Random Projection algorithm. In particular, the method proposed in [32] is data-dependent and, unlike the original Random Projection algorithm, requires an expensive training phase. In addition, this algorithm is not compatible with the database-friendly distribution proposed by Achlioptas [1] and its distance-preserving capabilities have not been analyzed empirically.…”