Graph Embedded Nonparametric Mutual Information for Supervised Dimensionality Reduction

Bouzas, Dimitrios; Arvanitopoulos, Nikolaos; Tefas, Anastasios

doi:10.1109/tnnls.2014.2329240

Cited by 30 publications

(11 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Here, we should note that several other methods which employ pairwise similarity/distance measures, e.g. [8], [13], [15]- [20], can be formulated using the Graph Embedding framework.…”

Section: A Graph Embeddingmentioning

confidence: 99%

Graph Embedding With Data Uncertainty

Laakom¹,

Raitoharju²,

Passalis³

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Spectral-based subspace learning is a common data preprocessing step in many machine learning pipelines. The main aim is to learn a meaningful low dimensional embedding of the data. However, most subspace learning methods do not take into consideration possible measurement inaccuracies or artifacts that can lead to data with high uncertainty. Thus, learning directly from raw data can be misleading and can negatively impact the accuracy. In this paper, we propose to model artifacts in training data using probability distributions; each data point is represented by a Gaussian distribution centered at the original data point and having a variance modeling its uncertainty. We reformulate the Graph Embedding framework to make it suitable for learning from distributions and we study as special cases the Linear Discriminant Analysis and the Marginal Fisher Analysis techniques. Furthermore, we propose two schemes for modeling data uncertainty based on pair-wise distances in an unsupervised and a supervised contexts.

show abstract

“…Here, we should note that several other methods which employ pairwise similarity/distance measures, e.g. [8], [13], [15]- [20], can be formulated using the Graph Embedding framework.…”

Section: A Graph Embeddingmentioning

confidence: 99%

Graph Embedding With Data Uncertainty

Laakom¹,

Raitoharju²,

Passalis³

et al. 2022

IEEE Access

View full text Add to dashboard Cite

show abstract

“…For instance, in the picture sharing community Flickr, there are billions of images and each can be annotated with textual labels selected from millions of candidates. In the community of neural networks and related learning systems, to handle the challenges, some works like [22]- [26] focus on feature dimension reduction or model simplification, while others like LI-MLC [21] focus on shrinking the label space. Here we follow the latter one.…”

Section: Responses To Reviewsmentioning

confidence: 99%

End-to-End Feature-Aware Label Space Encoding for Multilabel Classification With Many Classes

Lin

Ding

Han

et al. 2018

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

To make the problem of multilabel classification with many classes more tractable, in recent years, academia has seen efforts devoted to performing label space dimension reduction (LSDR). Specifically, LSDR encodes high-dimensional label vectors into low-dimensional code vectors lying in a latent space, so as to train predictive models at much lower costs. With respect to the prediction, it performs classification for any unseen instance by recovering a label vector from its predicted code vector via a decoding process. In this paper, we propose a novel method, namely End-to-End Feature-aware label space Encoding (EFE), to perform LSDR. Instead of requiring an encoding function like most previous works, EFE directly learns a code matrix formed by code vectors of the training instances in an end-to-end manner. Another distinct property of EFE is its feature awareness attributable to the fact that the code matrix is learned by jointly maximizing the recoverability of the label space and the predictability of the latent space. Based on the learned code matrix, EFE further trains predictive models to map instance features into code vectors, and also learns a linear decoding matrix for efficiently recovering the label vector of any unseen instance from its predicted code vector. Theoretical analyses show that both the code matrix and the linear decoding matrix in EFE can be efficiently learned. Moreover, similar to previous works, EFE can be specified to learn an encoding function. And it can also be extended with kernel tricks to handle nonlinear correlations between the feature space and the latent space. Comprehensive experiments conducted on diverse benchmark data sets with many classes show consistent performance gains of EFE over the state-of-the-art methods.

show abstract

“…The effect of the curse of dimensionality on proximity-based algorithms, such as k-means clustering or k-NN classification, has been extensively studied in the past (Aggarwal et al 2001;Beyer et al 1999). The observation that distances become increasingly meaningless as the dimensionality of data increases has lead researchers to the development of more robust proximity measures, such as metric-learning approaches (Xing et al 2003), as well as towards dimensionality reduction methods (Bouzas et al 2015;Nikitidis et al 2014;Passalis and Tefas 2017).…”

Section: Related Workmentioning

confidence: 99%

Self-supervised autoencoders for clustering and classification

Nousi

Tefas

2018

Evolving Systems

Self Cite

View full text Add to dashboard Cite

Clustering techniques aim at finding meaningful groups of data samples which exhibit similarity with regards to a set of characteristics, typically measured in terms of pairwise distances. Due to the so-called curse of dimensionality, i.e., the observation that high-dimensional spaces are unsuited for measuring distances, distance-based clustering techniques such as the classic k-means algorithm fail to uncover meaningful clusters in high-dimensional spaces. Thus, dimensionality reduction techniques can be used to greatly improve the performance of such clustering methods. In this work, we study Autoencoders as Deep Learning tools for dimensionality reduction, and combine them with k-means clustering to learn low-dimensional representations which improve the clustering performance by enhancing intra-cluster relationships and suppressing intercluster ones, in a self-supervised manner. In the supervised paradigm, distance-based classifiers may also greatly benefit from robust dimensionality reduction techniques. The proposed method is evaluated via multiple experiments on datasets of handwritten digits, various objects and faces, and is shown to improve external cluster quality measuring criteria. A fully supervised counterpart is also evaluated on two face recognition datasets, and is shown to improve the performance of various lightweight classifiers, allowing their use in real-time applications on devices with limited computational resources, such as Unmanned Aerial Vehicles (UAVs).

show abstract

Graph Embedded Nonparametric Mutual Information for Supervised Dimensionality Reduction

Cited by 30 publications

References 29 publications

Graph Embedding With Data Uncertainty

Graph Embedding With Data Uncertainty

End-to-End Feature-Aware Label Space Encoding for Multilabel Classification With Many Classes

Self-supervised autoencoders for clustering and classification

Contact Info

Product

Resources

About