Confidence estimation for t-SNE embeddings using random forest

Abstract

Dimensionality reduction algorithms are commonly used for reducing the dimension of multi-dimensional data to visualize them on a standard display. Although many dimensionality reduction algorithms such as the t-distributed Stochastic Neighborhood Embedding aim to preserve close neighborhoods in low-dimensional space, they might not accomplish that for every sample of the data and eventually produce erroneous representations. In this study, we developed a supervised confidence estimation algorithm for detecting erroneous samples in embeddings. Our algorithm generates a confidence score for each sample in an embedding based on a distance-oriented score and a random forest regressor. We evaluate its performance on both intra- and inter-domain data and compare it with the neighborhood preservation ratio as our baseline. Our results showed that the resulting confidence score provides distinctive information about the correctness of any sample in an embedding compared to the baseline. The source code is available at https://github.com/gsaygili/dimred.

Effect of distance measures on confidences of t-SNE embeddings and its implications on clustering for scRNA-seq data

“…In this table, the total number of erroneous samples for each dataset was also provided since the ratio of the correctly detected erroneous samples could be useful for interpreting the performance differences between the datasets. In most cases, our predictions generally showed higher concordance to the ground truth erroneous samples in comparison to Yigin et al 8 and the NPR score. It is noteworthy that the success of detecting erroneous samples has increased at a much higher rate than 8 particularly in the lowest 10 RF-scored samples, even occasionally a success of 10 out of 10 had been attained.…”

“…The distance measures are presented in Table 1 . In accordance with 8 , we trained an RF regressor on the training sets of the AMB18 and Baron Human datasets by using each distance measure individually as a feature. Next, we evaluate the performance of each distance measure with intra-dataset experiments.…”

“…We trained an RF regressor by using grid search for fine-tuning on the two different training sets, AMB18 and Baron Human, together with their generated confidence scores, and used this algorithm to predict the confidence scores of the other datasets (inter- and intra-datasets). In this study, we examined how well our suggested model performed against the model created on two distinct domains and different distance measures by Yigin et al 8 . We kept the same split for train and test sets (ratio of 80/20) in the shared datasets for both experiments to have a fair comparison, and we kept the same performance evaluation method.…”

“…Ranking-based metrics 5 , 6 focus on retaining local neighborhood rankings in high and low dimensions instead of considering the preservation of ground truth target labels (label-based). There are also some label-based error detection and confidence estimation methods that have been developed specifically for t-SNE embeddings 7 , 8 , in a similar way to those in many other domains such as medical image registration 9 , 10 and stereo matching 11 . What makes the label-based confidence estimation algorithm 8 unique is that it generates confidence scores for each and every sample in a t-SNE embedding with a supervised Random Forest (RF) regression algorithm based on target class labels.…”

“…There are also some label-based error detection and confidence estimation methods that have been developed specifically for t-SNE embeddings 7 , 8 , in a similar way to those in many other domains such as medical image registration 9 , 10 and stereo matching 11 . What makes the label-based confidence estimation algorithm 8 unique is that it generates confidence scores for each and every sample in a t-SNE embedding with a supervised Random Forest (RF) regression algorithm based on target class labels. The six different distance measures utilized as features for the regressor were not chosen domain-specifically, but rather common ones.…”