Deep Metric Learning with BIER: Boosting Independent Embeddings Robustly

Abstract

Learning similarity functions between image pairs with deep neural networks yields highly correlated activations of embeddings. In this work, we show how to improve the robustness of such embeddings by exploiting the independence within ensembles. To this end, we divide the last embedding layer of a deep network into an embedding ensemble and formulate the task of training this ensemble as an online gradient boosting problem. Each learner receives a reweighted training sample from the previous learners. Further, we propose two loss functions which increase the diversity in our ensemble. These loss functions can be applied either for weight initialization or during training. Together, our contributions leverage large embedding sizes more effectively by significantly reducing correlation of the embedding and consequently increase retrieval accuracy of the embedding. Our method works with any differentiable loss function and does not introduce any additional parameters during test time. We evaluate our metric learning method on image retrieval tasks and show that it improves over state-of-the-art methods on the CUB-200-2011, Cars-196, Stanford Online Products, In-Shop Clothes Retrieval and VehicleID datasets. Therefore, our findings suggest that by dividing deep networks at the end into several smaller and diverse networks, we can significantly reduce overfitting.

Divide and Conquer the Embedding Space for Metric Learning

“…network representation is powerful enough to approximate arbitrary continuous functions [19], in practice this often leads to poor local minima and overfitting. This is partially due to an inefficient usage of the embedding space [34,36] and an attempt to directly fit a single distance metric to all available data [38,28,29].…”

Deep Metric Learning using Similarities from Nonlinear Rank Approximations

“…In summary our method is only outperformed by attention based ensembles like ABE [13]. As mentioned before, HDC, A-BIER [19], ABE-8 and Proxy-NCA [18] use more complex/multiple network architectures. These networks are applicable in combination with our loss function and will lead to improved retrieval results.…”

Hybrid-Attention Based Decoupled Metric Learning for Zero-Shot Image Retrieval

“…Comparison with State-of-the-art: In order to highlight the significance of our decoupling idea for zero-shot retrieval, we compare DeML with some remarkable embed- From these tables, one can observe that our baseline(U512) can only achieve the general performances, and the performances of traditional ideas of loss-designing and samples-mining are almost on par with each other, while, by explicitly intensifying both the discrimination and diversity within the deep metric, our DeML can significantly improve the performances over baseline model and outperforms other state-of-the-art methods by a noteworthy margin, demonstrating the necessity of our explicit enhancement for discrimination and generalization via the decoupling idea. Moreover, different from the listed ensemble methods [43,24,25,13], DeML has clear objects of jointly mitigating the aforementioned issues that are vital to ZSIR, and thus can easily surpass them. Worthy of mention is that, we find DeML(I=2,J=4) is enough for Stanford Online Products dataset, since as in Fig.4 the second scale has been capable of localizing the discriminative regions.…”

“…Yuan et al [43] employ multiple layers at different depths for hard-aware samples mining and then cascade the learned embeddings together. Opitz et at [24,25] adopt the online gradients boosting and optimize different learners with the reweighted data. Kim et at [13] try to increase the feature diversity via contrastive loss but ignore the importance of learning discriminative metric in ZSIR task.…”

“…Additionally, HDC [43] employs the cascaded models and selects hard-samples from different levels and models. BIER loss [24,25] adopts the online gradients boosting methods. Although these two methods try to improve the performances by resorting to the ensemble idea, they can only learn from the undiscriminating input and also suffer from the partial learning behavior, as a result, the discrimination and generalization of holistic features are still limited.…”