A General Framework for Dimensionality-Reducing Data Visualization Mapping

Abstract

In recent years, a wealth of dimension-reduction techniques for data visualization and preprocessing has been established. Nonparametric methods require additional effort for out-of-sample extensions, because they provide only a mapping of a given finite set of points. In this letter, we propose a general view on nonparametric dimension reduction based on the concept of cost functions and properties of the data. Based on this general principle, we transfer nonparametric dimension reduction to explicit mappings of the data manifold such that direct out-of-sample extensions become possible. Furthermore, this concept offers the possibility of investigating the generalization ability of data visualization to new data points. We demonstrate the approach based on a simple global linear mapping, as well as prototype-based local linear mappings. In addition, we can bias the functional form according to given auxiliary information. This leads to explicit supervised visualization mappings with discriminative properties comparable to state-of-the-art approaches.

Parametric nonlinear dimensionality reduction using kernel t-SNE

“…Popular models such as PCA, SOM, its probabilistic counterparts the probabilistic PCA or the generative topographic mapping (GTM), and encoder frameworks such as deep autoencoder networks fall under the first, generative framework [17,32,4]. The second framework can cover diverse modern non-parametric approaches such as Isomap, MVU, LLE, SNE, or t-SNE, as recently demonstrated in the overview [6].…”

“…Many alternative non-parametric techniques proposed in the literature have a very similar structure, as pointed out in [6]: They extract a characteristic of the data points x i and try to find projections y i such that the corresponding characteristics are as close as possible as measured by some cost function. [6] summarizes some of today's most popular dimensionality reduction methods this way. In the following, we will exemplarily consider the alternatives maximum variance unfolding (MVU), locally linear embedding (LLE), and Isomap.…”

“…We fix a parametric form x → f w (x) = y and optimize the parameters of f w instead of the projection coordinates. Such an extension of non-parametric approaches to a parametric version has been proposed in [29,6,9] in different forms. In [29], f w takes the form of deep-autoencoder networks, which are trained in two steps: first, the deep auto-encoder is trained in a standard way to encode the given examples; afterwards, parameters are fine tuned such that the t-SNE cost function is optimized when plugging the images of given data points into the mapping.…”

“…In the last years, a huge variety of diverse alternative DR techniques has emerged, including popular algorithms such as locally linear embedding (LLE), Isomap, Isotop, maximum variance unfolding (MVU), Laplacian eigenmaps, neighborhood retrieval visualizer, maximum entropy unfolding, t-distributed stochastic neighbor embedding (t-SNE), and many others [23,26,35,3,31,33], see e.g. [32,33,17,6] for overviews. These methods belong to nonlinear DR techniques, enabling the correct visualization of data which lie on curved manifolds or which incorporate clusters of complex shape, as is often the case for real-life examples, thus opening the way towards a visual inspection of nonlinear phenomena in the given data.…”

Using Discriminative Dimensionality Reduction to Visualize Classifiers

“…Training takes place by tuning the projections y i such that a certain criterion is optimized: usually, the structure in the data space as defined by x i and the structure of the projections y i are measured and compared using some suitable cost function. An overview about a generic formalization of different popular non-parametric DR techniques as cost function optimization can be found in [3]. We will exemplarily investigate the following three popular techniques:…”