A Kernel-Learning Approach to Semi-supervised Clustering with Relative Distance Comparisons

Amid, Ehsan; Gionis, Aristides; Ukkonen, Antti

doi:10.1007/978-3-319-23528-8_14

Cited by 12 publications

(7 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, LDA fails if the conditional densities of the data are not multivariate Gaussian [42]. Therefore, we will consider the most reliable method, such as a relative distance comparison method [43] or triplet method [44] to preserve discriminant information of both domains.…”

Section: Discussionmentioning

confidence: 99%

Linear Discriminant Analysis via Pseudo Labels: A Unified Framework for Visual Domain Adaptation

Sanodiya

Yao

2020

IEEE Access

View full text Add to dashboard Cite

This paper deals with the problem of visual domain adaptation in which source domain labeled data is available for training, but the target domain unlabeled data is available for testing. Many recent domain adaptation methods merely concentrate on extracting domain-invariant features via minimizing the distributional and geometrical divergence between domains simultaneously while ignoring within-class and between-class structure properties, especially for the target domain due to the unavailability of labeled data. We propose Linear Discriminant Analysis via Pseudo Labels (LDAPL), a unified framework for visual domain adaptation that can tackle these two issues together. LDAPL is to learn a domain-invariant features across both domains with preserving important properties such as minimizing the shift between domains both statistically and geometrically, retaining the original similarity of data samples, maximizing the target domain variance, and minimizing the within-class and maximizing the between-class properties of both domains. Specifically, LDAPL preserves the target domain discriminative information (or within-class and between-class properties) using pseudo labels, and these pseudo labels are refined until convergence. Extensive experiment on several visual cross-domain benchmarks, including Office+Caltech10 with all three types of features (such as Speeded Up Robust Features (SURF), Deep Convolutional Activation Feature (DeCAF 6 ), and Visual Geometry Group-Fully Connected layer (VGG-FC 6 ) features), COIL20 (Columbia Object Image Library), digit, and PIE (Pose, Illumination, and Expression), LDAPL achieved average accuracies of 79.11%, 99.72%, 79.0%, and 84.50%, respectively. Comparative results on several visual cross-domain classification tasks verify that LDAPL can significantly outperform the state-of-theart primitive and domain adaptation methods. Specifically, LDAPL gains over baseline Joint Geometrical and Statistical Alignment (JGSA) method with 6.6%, 5.3%, 6.3%, and 44.93% average accuracies on Office+Caltech10 (SURF, DeCAF 6 , and VGG-FC 6 ), COIL20, digit, and PIE, respectively.

show abstract

Section: Discussionmentioning

confidence: 99%

Linear Discriminant Analysis via Pseudo Labels: A Unified Framework for Visual Domain Adaptation

Sanodiya

Yao

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…In [21], Mahadevan et al considered the Riemannian geometry of covariance matrices to minimize geometrical and statistical shifts between domains while learning a metric. Amid et al [22], proposed an algorithm that learns a kernel matrix using the log-determinant divergence that is subject to a set of relative distance constraints. Dai et al proposed an alg orithm called EigenTransfer [23] which learns the spectra of a graph that is obtained from the learning task to obtain eigenvectors that are able to accurately capture the structure of the graph.…”

Section: Related Workmentioning

confidence: 99%

A New Transfer Learning Algorithm in Semi-Supervised Setting

et al. 2019

View full text Add to dashboard Cite

Transfer Learning is an effective method of dealing with real-world problems where the training and test data are drawn from different distributions. Transfer learning methods use a labeled source domain to boost the task in a target domain that may be unsupervised or semi-supervised. However, the previous transfer learning algorithms use Euclidean distance or Mahalanobis distance formula to represent the relationships between instances and to try and capture the geometry of the manifold. In many real-world scenarios, this is not enough and these functions fail to capture the intrinsic geometry of the manifold that the data exists in. In this paper, we propose a transfer learning framework called Semi-Supervised Metric Transfer Learning with Relative Constraints (SSMTR), that uses distance metric learning with a set of relative distance constraints that capture the similarities and dissimilarities between the source and the target domains better. In SSMTR, instance weights are learned for different domains which are then used to reduce the domain shift while a Relative Distance metric is learned in parallel. We have developed SSMTR for classification problems as well, and have conducted extensive experiments on several real-world datasets; particularly, the PIE Face, Office-Caltech, and USPS-MNIST datasets to verify the accuracy of our proposed algorithm when compared to the current transfer learning algorithms.

show abstract

“…The distances between data points in ML constraints should be minimum, while the distances between those in CL constraints must be large. But, according to [25,26], the relative-distance constraints like inequality constraints set (C neq ) and equality constraintsset (C eq ) are particularly effective in expressing structures at a finer level of detail when compared to ML and CL constraints. Therefore, in our proposed work, we consider relative distance constraints as side information additionally to the feature vector for preserving the discriminate information.…”

Section: Introductionmentioning

confidence: 99%

“…In relative distance learning, the objective is to look for a novel distance function between data points, taking into account feature vector and relative distance constraints. We depart from existing literature [25] by eliciting every relative distance comparison constraint with a question "Which one of the data points "i", "j", and "k" is the least similar(or dissimilar) to the other two data points? "…”

Section: Introductionmentioning

confidence: 99%

Unsupervised Transfer Learning via Relative Distance Comparisons

Sanodiya

Yao

2020

IEEE Access

View full text Add to dashboard Cite

Primitive machine learning method such as Support Vector Machine (SVM) or k-Nearest Neighbor (k-NN) faces a major challenge when its training and test data is distributed with large-scale variations in lighting conditions, color, backgrounds, size, etc. The variation may be because training and testing data can come from related but some other domains. Considerable efforts have been made in the development of transfer learning methods. However, most current work focuses only on the following goals or objectives: i)Preservation of source domain discriminative information with Linear Discriminant Analysis (LDA); ii)Maximization of target domain variance; iii) Subspace alignment; iv) Minimization of marginal and conditional distribution by using the Maximum Mean Discrepancy (MMD) criterion;v)Preservation of original similarity of the data samples. Current approaches to preserve source domain discriminant information can easily misclassify the target domain samples which are distributed near the edge of the cluster. In order to overcome the limitations of existing transfer learning methods, we propose a novel Unsupervised Transfer Learning Via Relative Distance Comparisons (UTRDC) method. UTRDC optimizes all the aforementioned objectives jointly with a common projection vector matrix for both domains as well as uses the relative distance constraints for better inter-class separability and intra-class compactness. Furthermore, we extend our proposed method UTRDC to a kernelized version to deal with non-linear separable datasets. Extensive experimentation on two real-world problems datasets (PIE face and Office+Caltech) has proven that the proposed methods outperform several approaches to non-transfer learning and transfer learning.

show abstract

A Kernel-Learning Approach to Semi-supervised Clustering with Relative Distance Comparisons

Cited by 12 publications

References 15 publications

Linear Discriminant Analysis via Pseudo Labels: A Unified Framework for Visual Domain Adaptation

Linear Discriminant Analysis via Pseudo Labels: A Unified Framework for Visual Domain Adaptation

A New Transfer Learning Algorithm in Semi-Supervised Setting

Unsupervised Transfer Learning via Relative Distance Comparisons

Contact Info

Product

Resources

About