Graph Embedding With Data Uncertainty

Abstract

Spectral-based subspace learning is a common data preprocessing step in many machine learning pipelines. The main aim is to learn a meaningful low dimensional embedding of the data. However, most subspace learning methods do not take into consideration possible measurement inaccuracies or artifacts that can lead to data with high uncertainty. Thus, learning directly from raw data can be misleading and can negatively impact the accuracy. In this paper, we propose to model artifacts in training data using probability distributions; each data point is represented by a Gaussian distribution centered at the original data point and having a variance modeling its uncertainty. We reformulate the Graph Embedding framework to make it suitable for learning from distributions and we study as special cases the Linear Discriminant Analysis and the Marginal Fisher Analysis techniques. Furthermore, we propose two schemes for modeling data uncertainty based on pair-wise distances in an unsupervised and a supervised contexts.

Non-Linear Spectral Dimensionality Reduction Under Uncertainty

“…Theorem 1 corresponds to the Representer theorem for our graph preserving criterion. It indicates how each input uncertainty, i.e., P i , contributes to the global solution of (10). Furthermore, it guarantees that there is always a solution in the linear span of the data points and simplifies the problem search space to a finite dimensional subspace of the original function space which is often infinite dimensional.…”

“…Recently, modeling of uncertainty has gained attention in machine learning community in general [3], [4], [5], [6] and in dimensionality reduction (DR) in particular [7], [8], [9], [10]. Various DR techniques have been proposed which consider data uncertainty and inaccuracies [11], [12], [13], [14], [15].…”

“…The main hypothesis of this paradigm is that certain data features are corrupted by noise [9]. In [10], several traditional linear DR techniques, e.g., LDA and Marginal Fisher Analysis (MFA), were extended to consider uncertainty. However, the main drawback of that approach is that uncertainty modeling is restricted to the Gaussian case, which is impractical for many applications.…”

“…As illustrated in Figure 1, we view our samples as probability distributions modeling the inaccuracies and uncertainties of the data, i.e., each data point has a corresponding distribution modeling the uncertainty of its features. Furthermore, unlike [10], NGEU is not restricted to Gaussian noise and is suitable to learn from a wide range of distributions 1 . This allows for a more flexible modeling of the uncertainty in the input space.…”

“…The proposed framework can be used to derive robust extensions of DR methods formulated with the kernelization of the original GE framework [27], including Kernel Principal Component Analysis (KPCA), Kernel Discriminant Analysis (KDA), and Kernel Marginal Fisher Analysis (KMFA). While it has been shown empirically that leveraging the uncertainty information via distributional data in dimensionality reduction setup improves the robustness of the model [10], no theoretical guarantees have been provided yet. To bridge this gap, in this paper, we theoretically analyze how incorporating the uncertainty information affects the generalization ability of the methods, based on the Rademacher complexity.…”