LoOp: Looking for Optimal Hard Negative Embeddings for Deep Metric Learning

Vasudeva, Bhavya; Deora, Puneesh; Bhattacharya, Sumanta; Pal, Umapada; Chanda, Sukalpa

doi:10.1109/iccv48922.2021.01046

Cited by 14 publications

(6 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…InfoNCE loss was identified to have the hardnessaware property, which is critical for optimization [64] and preventing collapse by instance de-correlation [1]. [15], [34], [36], [49], [62], [66], [69] have demonstrated that hard negative samples mining strategies can be beneficial for better performance over the baselines. Notably, [65] identified CL form alignment and uniformity of feature space which benefits downstream tasks.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

DimCL: Dimensional Contrastive Learning for Improving Self-Supervised Learning

et al. 2023

View full text Add to dashboard Cite

Self-supervised learning (SSL) has gained remarkable success, for which contrastive learning (CL) plays a key role. However, the recent development of new non-CL frameworks has achieved comparable or better performance, prompting researchers to enhance these frameworks further. Assimilating CL into non-CL frameworks has been thought to be beneficial, but empirical evidence indicates no visible improvements. In view of that, this paper proposes a strategy of performing CL along the dimensional direction instead of along the batch direction as done in convention contrastive learning, named Dimensional Contrastive Learning (DimCL). DimCL aims to enhance the feature diversity, and it can serve as a regularizer to prior SSL frameworks. DimCL has been found to be effective, and the hardness-aware property is identified as a critical reason for its success. Extensive experimental results reveal that assimilating DimCL into SSL frameworks leads to performance improvement by a non-trivial margin on various datasets and backbone architectures.

show abstract

Section: Related Workmentioning

confidence: 99%

“…In the view of optimization, the hardness-aware property puts more weight into optimizing negative pairs that have high similarities. This way is influenced by hard examples mining and has proven to be effective [4], [36], [49], [62], [66], [72].…”

Section: B Hardness-aware Property In Dimclmentioning

confidence: 99%

DimCL: Dimensional Contrastive Learning for Improving Self-Supervised Learning

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Most of these methods ignored inter-class diversity and were based on costly hard mining strategies. Further, Vasudeva et al [30] looked for optimal hard mining. In addition, Xuan et al [31] showed the importance of intra-class variance in learning embedding.…”

Section: Related Workmentioning

confidence: 99%

Learning Noise-Assisted Robust Image Features for Fine-Grained Image Retrieval

Kumar¹,

Petwal²,

Gairola³

et al. 2023

Computer Systems Science and Engineering

View full text Add to dashboard Cite

Fine-grained image search is one of the most challenging tasks in computer vision that aims to retrieve similar images at the fine-grained level for a given query image. The key objective is to learn discriminative fine-grained features by training deep models such that similar images are clustered, and dissimilar images are separated in the low embedding space. Previous works primarily focused on defining local structure loss functions like triplet loss, pairwise loss, etc. However, training via these approaches takes a long training time, and they have poor accuracy. Additionally, representations learned through it tend to tighten up in the embedded space and lose generalizability to unseen classes. This paper proposes a noise-assisted representation learning method for fine-grained image retrieval to mitigate these issues. In the proposed work, class manifold learning is performed in which positive pairs are created with noise insertion operation instead of tightening class clusters. And other instances are treated as negatives within the same cluster. Then a loss function is defined to penalize when the distance between instances of the same class becomes too small relative to the noise pair in that class in embedded space. The proposed approach is validated on CARS-196 and CUB-200 datasets and achieved better retrieval results (85.38% recall@1 for CARS-196% and 70.13% recall@1 for CUB-200) compared to other existing methods.

show abstract

“…Deep metric learning aims to develop a similarity metric superior to traditional Euclidean distance by focusing on the derivation of high-dimensional data representations [1]. Contemporary research underscores the criticality of designing batch samplers [2][3][4] and employing online triplet mining [5][6][7][8], with hard negative mining schemes [9][10][11] traditionally serving as the preferred choice, proving their crucial role in significantly refining the similarity metric in deep metric learning. For an understanding of key concepts related to hard example mining and Batch Samplers, see the Section 1.…”

Section: Introductionmentioning

confidence: 99%

“…Challenge 2: Online Triplet Mining overlooks the issue of excessively loose clusters caused by outliers serving as anchors. The Online Triplet Mining (OTM) scheme, proposed by Triplet Loss [5], operates on triplets of examples (anchor, positive, negative) and generally aims to minimize the distance between the anchor and the positive example while maximizing the distance between the anchor and the negative example within the mini-batch [5,7,8,25]. Nonetheless, when the anchor is an outlier, it can lead to overly loose clusters, which are not preferable in deep learning due to their potential detriment to model classification.…”

Section: Introductionmentioning

confidence: 99%

Prototype-Based Support Example Miner and Triplet Loss for Deep Metric Learning

et al. 2023

View full text Add to dashboard Cite

Deep metric learning aims to learn a mapping function that projects input data into a high-dimensional embedding space, facilitating the clustering of similar data points while ensuring dissimilar ones are far apart. The most recent studies focus on designing a batch sampler and mining online triplets to achieve this purpose. Conventionally, hard negative mining schemes serve as the preferred batch sampler. However, most hard negative mining schemes search for hard examples in randomly selected mini-batches at each epoch, which often results in less-optimal hard examples and thus sub-optimal performances. Furthermore, Triplet Loss is commonly adopted to perform online triplet mining by pulling the hard positives close to and pushing the negatives away from the anchor. However, when the anchor in a triplet is an outlier, the positive example will be pulled away from the centroid of the cluster, thus resulting in a loose cluster and inferior performance. To address the above challenges, we propose the Prototype-based Support Example Miner (pSEM) and Triplet Loss (pTriplet Loss). First, we present a support example miner designed to mine the support classes on the prototype-based nearest neighbor graph of classes. Following this, we locate the support examples by searching for instances at the intersection between clusters of these support classes. Second, we develop a variant of Triplet Loss, referred to as a Prototype-based Triplet Loss. In our approach, a dynamically updated prototype is used to rectify outlier anchors, thus reducing their detrimental effects and facilitating a more robust formulation for Triplet Loss. Extensive experiments on typical Computer Vision (CV) and Natural Language Processing (NLP) tasks, namely person re-identification and few-shot relation extraction, demonstrated the effectiveness and generalizability of the proposed scheme, which consistently outperforms the state-of-the-art models.

show abstract

LoOp: Looking for Optimal Hard Negative Embeddings for Deep Metric Learning

Cited by 14 publications

References 28 publications

DimCL: Dimensional Contrastive Learning for Improving Self-Supervised Learning

DimCL: Dimensional Contrastive Learning for Improving Self-Supervised Learning

Learning Noise-Assisted Robust Image Features for Fine-Grained Image Retrieval

Prototype-Based Support Example Miner and Triplet Loss for Deep Metric Learning

Contact Info

Product

Resources

About