Grassmannian Learning: Embedding Geometry Awareness in Shallow and Deep Learning

Abstract

Modern machine learning algorithms have been adopted in a range of signal-processing applications spanning computer vision, natural language processing, and artificial intelligence. Many relevant problems involve subspace-structured features, orthogonality constrained or low-rank constrained objective functions, or subspace distances. These mathematical characteristics are expressed naturally using the Grassmann manifold. Unfortunately, this fact is not yet explored in many traditional learning algorithms. In the last few years, there have been growing interests in studying Grassmann manifold to tackle new learning problems. Such attempts have been reassured by substantial performance improvements in both classic learning and learning using deep neural networks. We term the former as shallow and the latter deep Grassmannian learning. The aim of this paper is to introduce the emerging area of Grassmannian learning by surveying common mathematical problems and primary solution approaches, and overviewing various applications. We hope to inspire practitioners in different fields to adopt the powerful tool of Grassmannian learning in their research. I. OVERVIEW OF GRASSMANN LEARNINGA Grassmann manifold refers to a space of subspaces embedded in a higher-dimensional vector space (e.g., the surface of a sphere in a 3D space). The mathematical notation arises in a variety of science and engineering applications in the areas of computer vision, statistical learning, wireless communications, and natural language processing. In visual recognition and classification tasks, the Grassmann manifold is used to model the invariant illumination or pose space [1,2]. In statistical learning, novel methods such as Grassmannian discriminant analysis [3,4,5] and clustering [6,7] are developed for processing data on the Grassmann manifold or exploiting tools from Grassmannian optimization to enhance learning performannce. In recommender systems, under low-rank or sparsity constraints, the problem of matrix completion can be solved using Grassmannian learning methods [8,9]. In wireless communications, Grassmannian packing can be applied to the design of space-time constellations [10,11,12] and that of limited feedback beamforming codebook [13,14]. In natural language processing, the Grassmann manifold can be used to model affine subspaces in document-specific language models J. Zhang is with the Dept. of Computer Science, and G. Zhu and K. Huang with the Dept. of Electrical & Electronic Engr., all

Manifold learning for coarse-graining atomistic simulations: Application to amorphous solids

“…, p, provides a measure of the distance between two points U A , U B ∈ G(p, n). A number of different distance measures are defined using the principal angles [59].…”

Deep Image Clustering with Tensor Kernels and Unsupervised Companion Objectives

“…We will stick to the Gaussian kernel, but use a distance function on the Grassmann manifold spanned by the respective matricizations (see Figure 2). Considering data matrices as points on the Grassmann manifold is the key concept in Grassmannian learning [25,26,27]. Results from this field indicate that using a distance function on the Grassmann manifold tends to improve performance of distancebased machine learning systems for tensor data [26,27].…”

“…Considering data matrices as points on the Grassmann manifold is the key concept in Grassmannian learning [25,26,27]. Results from this field indicate that using a distance function on the Grassmann manifold tends to improve performance of distancebased machine learning systems for tensor data [26,27].…”

Manifold learning-based polynomial chaos expansions for high-dimensional surrogate models

“…Dimensionality reduction is achieved using the Grassmanian diffusion maps (GDMaps) technique introduced in [50], which identifies a latent representation of the dataset on a lower-dimensional manifold via a two-step procedure. In the first step, high-dimensional data (model solutions) are projected onto an orthonormal matrix manifold called the Grassmann manifold [51,52] that defines the subspace structure of the data. In the second step the diffusion maps (DMaps) method is employed to unfold the underlying nonlinear geometry of the data on the Grassmann manifold onto a diffusion manifold.…”