Nonlinear Dimension Reduction via Local Tangent Space Alignment

Abstract

Nonlinear manifold learning from unorganized data points is a very challenging unsupervised learning and data visualization problem with a great variety of applications. In this paper we present a new algorithm for manifold learning and nonlinear dimension reduction. Based on a set of unorganized data points sampled with noise from the manifold, we represent the local geometry of the manifold using tangent spaces learned by fitting an affine subspace in a neighborhood of each data point. Those tangent spaces are aligned to give the internal global coordinates of the data points with respect to the underlying manifold by way of a partial eigendecomposition of the neighborhood connection matrix. We present a careful error analysis of our algorithm and show that the reconstruction errors are of second-order accuracy. We illustrate our algorithm using curves and surfaces both in 2D/3D and higher dimensional Euclidean spaces, and 64-by-64 pixel face images with various pose and lighting conditions. We also address several theoretical and algorithmic issues for further research and improvements. 1. Introduction. Many high-dimensional data in real-world applications can be modeled as data points lying close to a low-dimensional nonlinear manifold. Discovering the structure of the manifold from a set of data points sampled from the mani-fold possibly with noise represents a very challenging unsupervised learning problem [2, 3, 4, 8, 9, 10, 13, 14, 15, 17, 18]. The discovered low-dimensional structures can be further used for classification, clustering, outlier detection and data visualization. Example low-dimensional manifolds embedded in high-dimensional input spaces include image vectors representing the same 3D objects under different camera views and lighting conditions, a set of document vectors in a text corpus dealing with a specific topic, and a set of 0-1 vectors encoding the test results on a set of multiple choice questions for a group of students [13, 14, 18]. The key observation is that the dimensions of the embedding spaces can be very high (e.g., the number of pixels for each images in the image collection, the number of terms (words and/or phrases) in the vocabulary of the text corpus, or the number of multiple choice questions in the test), the intrinsic dimensionality of the data points, however, are rather limited due to factors such as physical constraints and linguistic correlations. Traditional dimension reduction techniques such as principal component analysis and factor analysis usually work well when the data points lie close to a linear (affine) subspace in the input space [7]. They can not, in general, discover nonlinear structures embedded in the set of data points. Recently, there have been much renewed interests in developing efficient algorithms for constructing nonlinear low-dimensional manifolds from sample data points in high-dimensional spaces, emphasizing simple algorithmic implementation and avoid

Manifold Reconstruction Using Tangential Delaunay Complexes

“…When the manifold is highly nonlinear, several more local techniques have attracted much attention in visual perception and many other areas of science. Among the prominent algorithms are Isomap [30], LLE [28], Laplacian eigenmaps [3], Hessian eigenmaps [18], diffusion maps [24,26], principal manifolds [31]. Most of those methods reduces to computing an eigendecomposition of some connection matrix.…”

Tangent-based manifold approximation with locally linear models

“…The importance of the tangent spaces for manifold related tasks has been lately recognized by many researchers. In [20], the authors use the tangent spaces to infer valid parametrizations of a manifold. Moreover, in [21], the tangent computed by a face image and its perturbed versions is used to capture the local geometry of the corresponding data manifold; faces are then classified based on the distances between their tangents.…”

Comparison of Dimension Reduction Methods for Database-Adaptive 3D Model Retrieval

“…Some of the examples of non-linear manifold learning algorithms are Isomap [25], Locally Linear Embedding (LLE) [21], Laplacian Eigenmaps (LE) [3], and Local Tangent Space Alignment (LTSA) [29].…”