Extendable and invertible manifold learning with geometry regularized autoencoders

Duque, Andrés; Morin, Sacha; Wolf, Guy; Moon, Kevin R.

doi:10.1109/bigdata50022.2020.9378049

Cited by 16 publications

(9 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many machine learning methods depend on some measure of pairwise similarity (which is usually unsupervised) including dimensionality reduction methods [17], [18], [19], [20], [21], [22], [23], spectral clustering [24], and any method involving the kernel trick such as SVM [25] and kernel PCA [26]. Random forest proximities can be used to extend many of these problems to a supervised setting and have been used for data visualization [27], [28], [29], [30], [31], outlier detection [30], [32], [33], [34], and data imputation [35], [36], [37], [38].…”

Section: Introductionmentioning

confidence: 99%

Geometry- and Accuracy-Preserving Random Forest Proximities

Rhodes¹,

Cutler²,

Moon³

2022

Preprint

View full text Add to dashboard Cite

Random forests are considered one of the best out-of-the-box classification and regression algorithms due to their high level of predictive performance with relatively little tuning. Pairwise proximities can be computed from a trained random forest which measure the similarity between data points relative to the supervised task. Random forest proximities have been used in many applications including the identification of variable importance, data imputation, outlier detection, and data visualization. However, existing definitions of random forest proximities do not accurately reflect the data geometry learned by the random forest. In this paper, we introduce a novel definition of random forest proximities called Random Forest-Geometry-and Accuracy-Preserving proximities (RF-GAP). We prove that the proximity-weighted sum (regression) or majority vote (classification) using RF-GAP exactly match the out-of-bag random forest prediction, thus capturing the data geometry learned by the random forest. We empirically show that this improved geometric representation outperforms traditional random forest proximities in tasks such as data imputation and provides outlier detection and visualization results consistent with the learned data geometry.

show abstract

Section: Introductionmentioning

confidence: 99%

Geometry- and Accuracy-Preserving Random Forest Proximities

Rhodes¹,

Cutler²,

Moon³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Some parameter variants of the above methods are proposed to improve generability and explainability. Topological autoencoders (TAE) [36] and Geometry Regularized AutoEncoders (GRAE) [11], for example, train autoencoders directly with local distance constraints in the input space. Ivis [49] propose a triplet loss function with distance as a constraint to train the neural network.…”

Section: Dimension Reduction and Data Visualizationmentioning

confidence: 99%

DMT-EV: An Explainable Deep Network for Dimension Reduction

Zang

Cheng

et al. 2024

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Dimension reduction (DR) is commonly utilized to capture intrinsic structure and transform high-dimensional data into low-dimensional space while retaining meaningful properties of original data. It is used in a wide variety of applications such as image recognition, single-cell sequencing analysis, and biomarker discovery. However, contemporary parametric-free and parametric DR techniques suffer from several major shortcomings, such as the inability to preserve both global and local features and the pool generalization performance. On the other hand, regarding explainability, it is crucial to comprehend the embedding process, especially the contribution of each feature to the embedding process while understanding how each feature affects the embedding results that identify critical components and helps diagnose the embedding process. To address these problems, we have developed a deep neural network method called EVNet which provides not only excellent performance in structural maintainability but also explainability to the DR therein. EVNet starts from data augmentation and with a manifold-based loss function to improve embedding performance. The explanation is based on saliency maps and is aimed to examine the trained EVNet parameters and contributions of components during the embedding process. The proposed techniques are integrated with a visual interface to help the user to adjust EVNet to achieve better DR performance and explainability. The interactive visual interface makes it easier to illustrate the data features, compare different DR techniques, and investigate the explainability of DR. An in-depth experimental comparison is provided which shows that EVNet consistently outperforms the state-of-the-art methods in both performance measures and explainability.

show abstract

“…Ding et al (2018;scvis) and Graving and Couzin (2020;VAE-SNE) describe VAE-derived dimensionality reduction algorithms based on the ELBO objective. Duque, Morin, Wolf, and Moon (2020; geometryregularized autoencoders) regularize an autoencoder with the PHATE (potential of heat-diffusion for affinity-based trajectory embedding) embedding algorithm (Moon et al, 2019). Szubert et al (2019;ivis) and Robinson (2020; differential embedding networks) make use of Siamese neural network architectures with structure-preserving loss functions to learn embeddings.…”

Section: Related Workmentioning

confidence: 99%

“…In principle, any embedding technique can be implemented parametrically by training a parametric model (e.g., a neural network) to predict embeddings from the original high-dimensional data (as in Duque et al, 2020). However, such a parametric embedding is limited in comparison to directly optimizing the algorithm's loss function.…”

Section: Comparisons With Indirect Parametric Embeddingsmentioning

confidence: 99%

Parametric UMAP Embeddings for Representation and Semisupervised Learning

McInnes²,

2021

View full text Add to dashboard Cite

UMAP is a nonparametric graph-based dimensionality reduction algorithm using applied Riemannian geometry and algebraic topology to find low-dimensional embeddings of structured data. The UMAP algorithm consists of two steps: (1) computing a graphical representation of a data set (fuzzy simplicial complex) and (2) through stochastic gradient descent, optimizing a low-dimensional embedding of the graph. Here, we extend the second step of UMAP to a parametric optimization over neural network weights, learning a parametric relationship between data and embedding. We first demonstrate that parametric UMAP performs comparably to its nonparametric counterpart while conferring the benefit of a learned parametric mapping (e.g., fast online embeddings for new data). We then explore UMAP as a regularization, constraining the latent distribution of autoencoders, parametrically varying global structure preservation, and improving classifier accuracy for semisupervised learning by capturing structure in unlabeled data.

show abstract

Extendable and invertible manifold learning with geometry regularized autoencoders

Cited by 16 publications

References 35 publications

Geometry- and Accuracy-Preserving Random Forest Proximities

Geometry- and Accuracy-Preserving Random Forest Proximities

DMT-EV: An Explainable Deep Network for Dimension Reduction

Parametric UMAP Embeddings for Representation and Semisupervised Learning

Contact Info

Product

Resources

About