Ranked List Loss for Deep Metric Learning

Abstract

The objective of deep metric learning (DML) is to learn embeddings that can capture semantic similarity and dissimilarity information among data points. Existing pairwise or tripletwise loss functions used in DML are known to suffer from slow convergence due to a large proportion of trivial pairs or triplets as the model improves. To improve this, ranking-motivated structured losses are proposed recently to incorporate multiple examples and exploit the structured information among them. They converge faster and achieve state-of-the-art performance. In this work, we unveil two limitations of existing ranking-motivated structured losses and propose a novel ranked list loss to solve both of them. First, given a query, only a fraction of data points is incorporated to build the similarity structure. Consequently, some useful examples are ignored and the structure is less informative. To address this, we propose to build a set-based similarity structure by exploiting all instances in the gallery. The learning setting can be interpreted as few-shot retrieval: given a mini-batch, every example is iteratively used as a query, and the rest ones compose the galley to search, i.e., the support set in few-shot setting. The rest examples are split into a positive set and a negative set. For every mini-batch, the learning objective of ranked list loss is to make the query closer to the positive set than to the negative set by a margin. Second, previous methods aim to pull positive pairs as close as possible in the embedding space. As a result, the intraclass data distribution tends to be extremely compressed. In contrast, we propose to learn a hypersphere for each class in order to preserve useful similarity structure inside it, which functions as regularisation. Extensive experiments demonstrate the superiority of our proposal by comparing with the state-of-the-art methods on the fine-grained image retrieval task. Our source code is available online: https://github.com/XinshaoAmos Wang/Ranked-List-Loss-for-DML.

Training deep retrieval models with noisy datasets: Bag exponential loss

“…3. List-based: these loss functions also make use of distances between pairs of images but, instead of computing the loss directly over them (as the pair-based do), distances are used to build intermediate list-like structures, such as soft-binning histograms, from which a final loss is derived [112,113,114,115]. Since these losses take into account (potentially) long lists containing all images in the training set, they circumvent the problem of locality, and provide a global analysis of the feature space.…”

Multi-level Deep Learning Vehicle Re-identification using Ranked-based Loss Functions

“…Multi-level embeddings, i.e. the generation of embedding by pooling information at different stages of the CNN, have also been used for extracting more robust feature representations in metric learning methods [12], [13]. However, in the vehicle ReID field, they have not been fully explored, despite the apparent benefit of multi-level embeddings to the problem.…”

“…Given triplets of samples projected to an embedding space, the purpose of triplet loss is to make the distance between the anchor and the negative point larger than the distance between the anchor and the positive point. Recently, group-based metric learning loss functions have been proposed [13], [18], [21], and achieve superior results in retrieval problems, since they incorporate more than three samples, and take into consideration valuable information among multiple data points. In several works [8], [14], classification loss is also used as a complement to similarity metric objectives, in order to increase the discriminative power of the network.…”

“…Based on [13], we employ RLL loss function, which aims to keep the distance between the query and a group of positive samples below a certain threshold, while separating the query from the negative samples by a margin. In our work, we explore and evaluate two variations of this loss function, that can be observed in Fig.…”

Adaptive Deep Metric Ensemble Learning with Consensus

“…Most contributions in deep metric learning have been devoted to the design of loss functions. The majority of them belong to pair-based functions, e.g., contrastive loss [6,12], triplet loss [7,13], N-pair loss [14], hierarchical triplet loss [15], ranked list loss [16], and multi-similarity loss with general pair weighting [17]. Besides, some loss functions adopt proxy mechanism, such as proxy Neighborhood Component Analysis (ProxyNCA) [18] and proxy anchor [19], to speedup the convergence of model training, where the optimization is carried out on the proxies of triplets, each of which involves an anchor point and similar/dissimilar proxy points.…”