It Takes Two to Tango: Mixup for Deep Metric Learning

Abstract

Metric learning involves learning a discriminative representation such that embeddings of similar classes are encouraged to be close, while embeddings of dissimilar classes are pushed far apart. State-of-the-art methods focus mostly on sophisticated loss functions or mining strategies. On the one hand, metric learning losses consider two or more examples at a time. On the other hand, modern data augmentation methods for classification consider two or more examples at a time. The combination of the two ideas is under-studied. In this work, we aim to bridge this gap and improve representations using mixup, which is a powerful data augmentation approach interpolating two or more examples and corresponding target labels at a time. This task is challenging because unlike classification, the loss functions used in metric learning are not additive over examples, so the idea of interpolating target labels is not straightforward. To the best of our knowledge, we are the first to investigate mixing examples and target labels for deep metric learning. We develop a generalized formulation that encompasses existing metric learning loss functions and modify it to accommodate for mixup, introducing Metric Mix, or Metrix. We show that mixing inputs, intermediate representations or embeddings along with target labels significantly improves representations and outperforms state-of-the-art metric learning methods on four benchmark datasets.

FPAdaMetric: False-Positive-Aware Adaptive Metric Learning for Session-Based Recommendation

“…Consequently, we introduce metric-learning regularization terms in the original problem (equation ( 6)), which we call FP-Metric. Metric-learning (Hoffer and Ailon 2015; Kaya and Bilge 2019) is well-known approach to learn appropriate representation via FP and positive samples in computer vision (Karpusha, Yun, and Fehervari 2020;Venkataramanan et al 2021) and audio (Chung et al 2020;Xu et al 2020) domains. Also, metric-learning is getting great attention recently due to high-performance in self-supervised and unsupervised approaches (Jaiswal et al 2021).…”

Recall@k Surrogate Loss with Large Batches and Similarity Mixup

“…Such a large batch is used for the first time in several of the employed image retrieval benchmarks. The batch size is virtually increased with another contribution, which is a computationally efficient mixup technique, called SiMix, that operates on the similarity scores instead of the embedding vectors as in prior work [55,13,12]. If the training set is not large enough and all of its classes are used to form a single batch, SiMix is shown essential to virtually increase the batch size and significantly boost the performance.…”

“…Linearly interpolating labels entails the risk of generating false negatives if the interpolation factor is close to 0 or 1. Such limitations are overcome in the work of Venkataramanan et al [55], which generalizes mixing examples from different classes for pairwise loss functions. The proposed SiMix approach differs from the aforementioned techniques as it operates on the similarity scores instead of the embedding vectors, making it computationally efficient.…”

“…Therefore, the minibatch is expanded to B∪ B by adding virtual examples without the need for explicit construction of the corresponding embeddings or estimation of the similarity via dot product; simple mixing of the corresponding pairwise scalar similarities is enough. SiMix reduces to mixing pairwise similarities due to the lack of re-normalization of the mixed embeddings, which is different to existing practice in prior work[55,12,13,21] and brings training efficiency benefits. Virtual examples are created only between examples of the same classes and are labeled according to the class of the original examples that are mixed.…”

Simple and Effective Balance of Contrastive Losses

“…Note that architectures, optimizers, and pooling layer differ slightly between the different implementations. For SOP and Landmarks, we use ResNet-18, GeM pooling, and SGD, [38]-R-GeM and [40] use ResNet-101, GeM pooling, and Adam, [33] uses BN-Inception, a linear projection, and RMSProp, [34] use Google LeNet, a linear projection, and SGD, and [47] use a Resnet50, a combination of average and max pooling followed by a linear projection, and AdamW. For TinyImageNet, we use ResNet-32 and SGD, and [36] uses ResNet-32 and Adam.…”