Exploiting Category Similarity-Based Distributed Labeling for Fine-Grained Visual Classification

Abstract

The fine-grained visual classification (FGVC) which aims to distinguish subtle differences among subcategories is an important computer vision task. However, one issue that limits model performance is the problem of diversity within subcategories. To this end, we propose a simple yet effective approach named category similarity-based distributed labeling (CSDL) to tackle this problem. Specifically, we first obtain the feature centers for various subcategories and utilize them to initialize the label distributions. Then we replace the ground-truth labels in a Deep Neural Network (DNN) with the distributed labels to calculate the loss and perform the optimization. Finally, the joint supervision of a softmax loss and a center loss is adopted to update the parameters of the DNN, the deep feature centers, and the distributed labels for learning discriminative deep features. Comprehensive experiments on three publicly available FGVC datasets demonstrate the superiority of our proposed approach.

Exploration of Class Center for Fine-Grained Visual Classification

“…AIR CUB CAR NAB B-CNN [60] 86.9 84.0 90.6 -MA-CNN [6] 89.9 86.5 92.8 -M2DRL [44] -87.2 93.3 -NTS-Net [43] 91.4 87.5 93.9 -Cross-X [9] 92.6 87.7 94.6 86.2 MGE-CNN [4] -88.5 93.9 86.7 ELP [61] 92.7 88.8 94.2 -DCL [48] 93.0 87.8 94.5 86.0 WSDAN [2] 93.0 89.4 94.5 87.9 SFFF [5] 93.1 85.4 94.4 -API-Net [62] 93.4 88.6 94.9 86.2 PMG [46] 93.4 89.6 95.1 -CDSL-DCL [13] 93.5 88.6 94.9 -CAL [59] 94.2 90.6 95.5 91.0 SIA-Net [35] 94. CNNs on the CUB and NAB datasets, but CNNs achieve better performances on the AIR and CAR datasets.…”

“…SCL aggregates representations of natural samples around the center point and increases the distance from manipulated samples to the center point, making it greater than from natural samples by a margin. CSDL [13] combines class centers and one-hot labels to generate soft labels. To measure the importance of samples in the same cluster, AdaMG [50] calculates the distances between these samples and their corresponding class centers.…”

“…One-hot labels are eligible for coarse-grained visual classification because of significant visual differences between coarse categories [13]. However for FGVC, models with hard labels pay attention to irrelevant features (e.g., background) or sample-specific noise to achieve high prediction confidence from these "hard" labels.…”

“…However, most of these methods rely only on cross entropy loss to obtain classification boundaries, which is insufficient to handle inter-class differences. In addition, some common visual classification methods introduce class center as representa-tions of whole classes, and reduce the distances between samples and class centers to reduce intra-class variances [12], [13], [14], [15]. In addition, Zhang et al proposed a feature aggregation scheme to resist intra-class variances [16].…”

“…Label smoothing (LS) uniformly reassigns partial confidence of the target class to the nontarget categories to generate soft labels [23]. Nevertheless, uniform confidence in nontarget categories ignores rich correlations between fine-grained categories [13], [24]. Compared with other nontarget classes, more similar nontarget classes should be assigned more confidence.…”