Fine-grained visual categorization via multi-stage metric learning

Abstract

Fine-grained visual categorization (FGVC) is to categorize objects into subordinate classes instead of basic classes. One major challenge in FGVC is the co-occurrence of two issues: 1) many subordinate classes are highly correlated and are difficult to distinguish, and 2) there exists the large intra-class variation (e.g., due to object pose). This paper proposes to explicitly address the above two issues via distance metric learning (DML). DML addresses the first issue by learning an embedding so that data points from the same class will be pulled together while those from different classes should be pushed apart from each other; and it addresses the second issue by allowing the flexibility that only a portion of the neighbors (not all data points) from the same class need to be pulled together. However, feature representation of an image is often high dimensional, and DML is known to have difficulty in dealing with high dimensional feature vectors since it would require O(d 2 ) for storage and O(d 3 ) for optimization. To this end, we proposed a multi-stage metric learning framework that divides the large-scale high dimensional learning problem to a series of simple subproblems, achieving O(d) computational complexity. The empirical study with FVGC benchmark datasets verifies that our method is both effective and efficient compared to the state-of-the-art FGVC approaches.

SoftTriple Loss: Deep Metric Learning Without Triplet Sampling

“…To alleviate this issue, we develop a strategy that sets a sufficiently large number of centers for each class at the beginning and then applies L 2,1 norm to obtain a compact set of centers. We demonstrate the proposed loss on the fine-grained visual categorization tasks, where capturing local clusters is essential for good performance [17].…”

Learning Similarity Conditions Without Explicit Supervision

“…Reasoning about the similarity between images or data of different modalities is an inherent challenge in computer vision. Beyond its prevalence in fundamental problems such as image-sentence retrieval [41,38], cross-domain image-matching [32,16], attribution learning [4,33] and visual categorization [29], it also has an increasingly prominent role in computer vision problems in the fashion and retail domains like outfit style modeling [14], fashion item retrieval and recommendation [10,22] and automatic capsule wardrobe generation [15]. Metric learning (the task of learning a distance function between features based on supervised similar/dissimilar pairs) is a common approach 1 https://github.com/rxtan2/ Learning-Similarity-Conditions Figure 1: We propose the SCE-Net model for learning multi-faceted similarity between images, such as compatibility of two fashion items.…”

REAPS: Towards Better Recognition of Fine-Grained Images by Region Attending and Part Sequencing

“…Besides, some approaches (e.g. [28], [37]) try to learn more robust representations via distance metric learning. [38] unifies deep CNN features with spatially weighted Fisher vectors to capture important details and eliminate background disturbance.…”