Learning Robust Distance Metric with Side Information via Ratio Minimization of Orthogonally Constrained L21-Norm Distances

Li, Kai; Brand, Lodewijk; Wang, Hua; Nie, Feiping

doi:10.24963/ijcai.2019/417

Cited by 12 publications

(7 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Existing approaches to improving the robustness of Mahalanobis distances can be categorized into four main types. The first type of method imposes structural assumption or regularization over M so as to avoid overfitting [25], [33]- [37]. Methods with structural assumption are proposed for classifying images and achieve robustness by exploiting the structural information of images; however, such information is generally unavailable in the symbolic datasets that will be studied in this paper.…”

Section: A Robust Metric Learningmentioning

confidence: 99%

Toward Certified Robustness of Distance Metric Learning

Yang

Guo

Dong

et al. 2024

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Metric learning aims to learn a distance metric such that semantically similar instances are pulled together while dissimilar instances are pushed away. Many existing methods consider maximizing or at least constraining a distance margin in the feature space that separates similar and dissimilar pairs of instances to guarantee their generalization ability. In this paper, we advocate imposing an adversarial margin in the input space so as to improve the generalization and robustness of metric learning algorithms. We first show that, the adversarial margin, defined as the distance between training instances and their closest adversarial examples in the input space, takes account of both the distance margin in the feature space and the correlation between the metric and triplet constraints. Next, to enhance robustness to instance perturbation, we propose to enlarge the adversarial margin through minimizing a derived novel loss function termed the perturbation loss. The proposed loss can be viewed as a datadependent regularizer and easily plugged into any existing metric learning methods. Finally, we show that the enlarged margin is beneficial to the generalization ability by using the theoretical technique of algorithmic robustness. Experimental results on 16 datasets demonstrate the superiority of the proposed method over existing state-of-the-art methods in both discrimination accuracy and robustness against possible noise.

show abstract

Section: A Robust Metric Learningmentioning

confidence: 99%

Toward Certified Robustness of Distance Metric Learning

Yang

Guo

Dong

et al. 2024

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

show abstract

“…To improve robustness to perturbation that is likely to exist in practice, many robust metric learning methods have been proposed, which can be categorized into three main types. The first type of methods imposes structural assumption or regularization over M so as to avoid overfitting [16,23,37,21,15,26,25]. However, structural information often exists in image datasets but is generally unavailable in the symbolic datasets studied in this paper.…”

Section: Related Workmentioning

confidence: 99%

Towards Certified Robustness of Metric Learning

Yang¹,

Guo²,

Dong³

et al. 2020

Preprint

View full text Add to dashboard Cite

Metric learning aims to learn a distance metric such that semantically similar instances are pulled together while dissimilar instances are pushed away. Many existing methods consider maximizing or at least constraining a distance "margin" that separates similar and dissimilar pairs of instances to guarantee their performance on a subsequent k-nearest neighbor classifier. However, such a margin in the feature space does not necessarily lead to robustness certification or even anticipated generalization advantage, since a small perturbation of test instance in the instance space could still potentially alter the model prediction. To address this problem, we advocate penalizing small distance between training instances and their nearest adversarial examples, and we show that the resulting new approach to metric learning enjoys a larger certified neighborhood with theoretical performance guarantee. Moreover, drawing on an intuitive geometric insight, the proposed new loss term permits an analytically elegant closed-form solution and offers great flexibility in leveraging it jointly with existing metric learning methods. Extensive experiments demonstrate the superiority of the proposed method over the state-of-the-arts in terms of both discrimination accuracy and robustness to noise.

show abstract

“…Liao et al [28] propose a two-stage metric learning method by combining 2,1 l -norm with LDA. To address the sensitive problem to outliers in [10], Liu et al [26] introduced the not-squared 2 l -norm instead of the squared 2 l -norm in objective function.…”

Section: Introductionmentioning

confidence: 99%

Adaptive Metric Learning for Dimensionality Reduction

Chen¹,

Wei²,

Long³

et al. 2022

JCC

View full text Add to dashboard Cite

Finding a suitable space is one of the most critical problems for dimensionality reduction. Each space corresponds to a distance metric defined on the sample attributes, and thus finding a suitable space can be converted to develop an effective distance metric. Most existing dimensionality reduction methods use a fixed pre-specified distance metric. However, this easy treatment has some limitations in practice due to the fact the pre-specified metric is not going to warranty that the closest samples are the truly similar ones. In this paper, we present an adaptive metric learning method for dimensionality reduction, called AML. The adaptive metric learning model is developed by maximizing the difference of the distances between the data pairs in cannot-links and those in must-links. Different from many existing papers that use the traditional Euclidean distance, we use the more generalized 2, p l -norm distance to reduce sensitivity to noise and outliers, which incorporates additional flexibility and adaptability due to the selection of appropriate p-values for different data sets. Moreover, considering traditional metric learning methods usually project samples into a linear subspace, which is overstrict. We extend the basic linear method to a more powerful nonlinear kernel case so that well capturing complex nonlinear relationship between data. To solve our objective, we have derived an efficient iterative algorithm. Extensive experiments for dimensionality reduction are provided to demonstrate the superiority of our method over state-of-the-art approaches.

show abstract

Learning Robust Distance Metric with Side Information via Ratio Minimization of Orthogonally Constrained L21-Norm Distances

Cited by 12 publications

References 1 publication

Toward Certified Robustness of Distance Metric Learning

Toward Certified Robustness of Distance Metric Learning

Towards Certified Robustness of Metric Learning

Adaptive Metric Learning for Dimensionality Reduction

Contact Info

Product

Resources

About