A Survey on Metric Learning for Feature Vectors and Structured Data

Abstract

The need for appropriate ways to measure the distance or similarity between data is ubiquitous in machine learning, pattern recognition and data mining, but handcrafting such good metrics for specific problems is generally difficult. This has led to the emergence of metric learning, which aims at automatically learning a metric from data and has attracted a lot of interest in machine learning and related fields for the past ten years. This survey paper proposes a systematic review of the metric learning literature, highlighting the pros and cons of each approach. We pay particular attention to Mahalanobis distance metric learning, a well-studied and successful framework, but additionally present a wide range of methods that have recently emerged as powerful alternatives, including nonlinear metric learning, similarity learning and local metric learning. Recent trends and extensions, such as semi-supervised metric learning, metric learning for histogram data and the derivation of generalization guarantees, are also covered. Finally, this survey addresses metric learning for structured data, in particular edit distance learning, and attempts to give an overview of the remaining challenges in metric learning for the years to come.

Algorithmic Robustness for Learning via $(ε, γ, τ)$-Good Similarity Functions

“…It turns out that the smaller the lipschitz constant of those similarity functions, the tighter the consistency bounds. This opens the door to new lines of research in metric learning (Bellet et al, 2013;2015) aiming at maximizing the (ǫ, γ, τ )-goodness of similarity functions s.t. ||A|| 2 is as small as possible (see pioneer works like Bellet et al (2012;2011)).…”

Context-Aware Visual Compatibility Prediction

“…We want the decoder to be a function that computes the probability that two nodes are connected. This scenario is known as metric learning [3], where the goal is to learn a notion of similarity or compatibility between data samples. It is relevant to note that similarity and compatibility are not exactly the same.…”

Approximate eigenvalue decompositions of orthonormal and symmetric transformations with a few Householder reflectors

“…Given labeled data points a problem that arises is constructing a distance metric such that points from the same class are "close" and points from different classes are "far". This is known as the distance metric learning problem [27]: given a training dataset, find a linear transformation of the input data such that points from the same class are concentrated while the separation between points of different classes increases. This technique has been shown to consistently produce improved results as compared to the Euclidean distance [28], [29], [30].…”