UnProjection: Leveraging Inverse-Projections for Visual Analytics of High-Dimensional Data

Abstract

Projection techniques are often used to visualize high-dimensional data, allowing users to better understand the overall structure of multi-dimensional spaces on a 2D screen. Although many such methods exist, comparably little work has been done on generalizable methods of inverse-projection -the process of mapping the projected points, or more generally, the projection space back to the original high-dimensional space. In this article we present NNInv, a deep learning technique with the ability to approximate the inverse of any projection or mapping. NNInv learns to reconstruct high-dimensional data from any arbitrary point on a 2D projection space, giving users the ability to interact with the learned high-dimensional representation in a visual analytics system. We provide an analysis of the parameter space of NNInv, and offer guidance in selecting these parameters. We extend validation of the effectiveness of NNInv through a series of quantitative and qualitative analyses. We then demonstrate the method's utility by applying it to three visualization tasks: interactive instance interpolation, classifier agreement, and gradient visualization.

Decision Boundary Visualization for Counterfactual Reasoning

“…Geometrically, the projection embeds a hyperplane in input space for the linear case and a hyper‐surface for the non‐linear case. This embedding can be evaluated by inversely projecting 2D points to input space creating a densely sampled representation [EAS*21] with small remaining uncertainty about n D values in‐between samples.…”

“…We use linear projections to create dense maps that inherently do not suffer from these problems. While linear projections have been considered for this application before [CCWH08], they were dismissed due to their poorer performance in cluster separation [SGH15, ERT19] and possible data point overlap as compared to non‐linear methods [EAS*21]. We show that by providing complementary interactive selection and interpretation tools, this weakness can be alleviated.…”

HyperNP: Interactive Visual Exploration of Multidimensional Projection Hyperparameters

“…Their method enables the exploration of the latent space of graph layouts. More recently, NNP [EHT20,EHFT20,EAS * 21] used deep learning to mimic any projection technique P by training on P ( D ′) for a small subset D ′ ⊂ D. In addition to providing fast computation, such approaches also have the ability to project out‐of‐sample data. Furthermore, due to the neural‐network based approach, inference computations are parameter free.…”

“…In this paper, we present HyperNP, a deep learning technique that approximates projections across hyperparameters to enable real‐time exploration of a projection method's hyperparameters. Similar to previous work on neural network projection (NNP) [EHT20,EAS * 21], HyperNP approximates the projection of a single technique (like t‐SNE) on a single dataset with the added benefit of out‐of‐sample projection. Both NNP and HyperNP can be thought of as surrogate models for projection techniques, which attempt to alleviate some of the computational burden by approximating the results.…”

Explainable Interactive Projections for Image Data

“…Some interactive DR methods create a bi-directional workflow where people can alter data in the high dimensional space to see the effect on the 2D location and vice versa [9,29]. Other works explore the idea of backwards (or inverse) projections that allow people to select locations in the 2D space and generate corresponding high-dimensional representations [13,21]. PEx-Image specifically targets image data, providing interactions for exploratory tasks, such as zooming into specific projection regions, displacing points to resolve overlapping and…”