Filtration and Distillation: Enhancing Region Attention for Fine-Grained Visual Categorization

Liu, Chuanbin; Xie, Hongtao; Zha, Zheng-Jun; Ma, Lingfeng; Yu, Lingyun; Zhang, Yongdong

doi:10.1609/aaai.v34i07.6822

Cited by 139 publications

(75 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Backbone Accuracy ResNet50 [13] ResNet50 84.5 GP-256 [31] VGG16 85.8 MaxEnt [8] DenseNet161 86.6 DFL-CNN [29] ResNet50 87.4 NTS-Net [34] ResNet50 87.5 Cross-X [20] ResNet50 87.7 DCL [3] ResNet50 87.8 CIN [10] ResNet101 88.1 DBTNet [37] ResNet101 88.1 ACNet [15] ResNet50 88.1 S3N [5] ResNet50 88.5 FDL [19] DenseNet161 89.1 PMG [7] ResNet50 89.6 API-Net [39] DenseNet161 90.0 StackedLSTM [11] GoogleNet 90.4 ViT [6] ViT-B_16 90.8 TransFG [12] ViT-B_16 91.7 FFVT ViT-B_16 91.6…”

Section: Methodsmentioning

confidence: 99%

Feature Fusion Vision Transformer for Fine-Grained Visual Categorization

Wang

Gao

2021

Preprint

View full text Add to dashboard Cite

The core for tackling the fine-grained visual categorization (FGVC) is to learn subtle yet discriminative features. Most previous works achieve this by explicitly selecting the discriminative parts or integrating the attention mechanism via CNN-based approaches. However, these methods enhance the computational complexity and make the model dominated by the regions containing the most of the objects. Recently, vision transformer (ViT) has achieved SOTA performance on general image recognition tasks. The self-attention mechanism aggregates and weights the information from all patches to the classification token, making it perfectly suitable for FGVC. Nonetheless, the classification token in the deep layer pays more attention to the global information, lacking the local and low-level features that are essential for FGVC. In this work, we propose a novel pure transformer-based framework Feature Fusion Vision Transformer (FFVT) where we aggregate the important tokens from each transformer layer to compensate the local, low-level and middle-level information. We design a novel token selection module called mutual attention weight selection (MAWS) to guide the network effectively and efficiently towards selecting discriminative tokens without introducing extra parameters. We verify the effectiveness of FFVT on three benchmarks where FFVT achieves the state-of-the-art performance.

show abstract

Section: Methodsmentioning

confidence: 99%

Feature Fusion Vision Transformer for Fine-Grained Visual Categorization

Wang

Gao

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, several studies 19–21 use only image labels to locate corresponding parts to compare their appearance, without compromising performance. This extensively alleviates the limitations of annotation.…”

Section: Related Workmentioning

confidence: 99%

Bin similarity‐based domain adaptation for fine‐grained image classification

Han

Zhang

Jia

2022

Int J of Intelligent Sys

View full text Add to dashboard Cite

Fine‐grained classification tasks are challenging because fine‐grained data sets are quite scarce. Thus, we utilized the domain adaptation method to migrate knowledge from large, labeled data sets to fine‐grained target data sets. We employed the bin similarity (BS) algorithm to measure and select the approximate domains from large‐scale data sets to the fine‐grained target domains. Source domain feature space was divided into multiple bins and the features of the target domains were sampled to fill the bins. The most similar domains were selected based on the similarity statistics of the sample features. We implemented the BS algorithm combined with the popular convolutional neural networks, pretrained the network on the selected similar subdata sets, and subsequently fine‐tuned it on the fine‐grained data sets. We evaluated the BS classification model on Stanford Dogs and Oxford Flower data sets, and the results showed improved BS classification performance compared with the state‐of‐the‐art domain adaptation methods, earth mover's distance, selective joint fine‐tuning, L2 with starting point, and domain similarity for transfer learning. Furthermore, BS is a pluggable module that boosts the performance of domain adaptation.

show abstract

“…Backbone Acc. (%) ResNet-50 [17] ResNet-50 84.5 RA-CNN [12] VGG-19 85.3 GP-256 [37] VGG-16 85.8 MaxExt [11] DenseNet-161 86.6 DFL-CNN [34] ResNet-50 87.4 NTS-Net [39] ResNet-50 87.5 Cross-X [26] ResNet-50 87.7 DCL [4] ResNet-50 87.8 CIN [14] ResNet-101 88.1 DBTNet [42] ResNet-101 88.1 ACNet [21] ResNet-50 88.1 S3N [8] ResNet-50 88.5 FDL [25] DenseNet-161 89.1 PMG [10] ResNet-50 89.6 API-Net [47] DenseNet-161 90.0 StackedLSTM [15] GoogleNet 90.4 MMAL-Net [40] ResNet-50 89.6 ViT [9] ViT-B_16 90.6 TransFG & PSM [16] ViT-B_16 90.9…”

Section: Methodsmentioning

confidence: 99%

RAMS-Trans: Recurrent Attention Multi-scale Transformer forFine-grained Image Recognition

Hu,

Jin,

Zhang

et al. 2021

Preprint

View full text Add to dashboard Cite

In fine-grained image recognition (FGIR), the localization and amplification of region attention is an important factor, which has been explored a lot by convolutional neural networks (CNNs) based approaches. The recently developed vision transformer (ViT) has achieved promising results on computer vision tasks. Compared with CNNs, Image sequentialization is a brand new manner. However, ViT is limited in its receptive field size and thus lacks local attention like CNNs due to the fixed size of its patches, and is unable to generate multi-scale features to learn discriminative region attention. To facilitate the learning of discriminative region attention without box/part annotations, we use the strength of the attention weights to measure the importance of the patch tokens corresponding to the raw images. We propose the recurrent attention multi-scale transformer (RAMS-Trans), which uses the transformer's self-attention to recursively learn discriminative region attention in a multi-scale manner. Specifically, at the core of our approach lies the dynamic patch proposal module (DPPM) guided region amplification to complete the integration of multiscale image patches. The DPPM starts with the full-size image patches and iteratively scales up the region attention to generate new patches from global to local by the intensity of the attention weights generated at each scale as an indicator. Our approach requires only the attention weights that come with ViT itself and can be easily trained end-to-end. Extensive experiments demonstrate that RAMS-Trans performs better than concurrent works, in addition to efficient CNN models, achieving state-of-the-art results on three benchmark datasets.

show abstract

Filtration and Distillation: Enhancing Region Attention for Fine-Grained Visual Categorization

Cited by 139 publications

References 22 publications

Feature Fusion Vision Transformer for Fine-Grained Visual Categorization

Feature Fusion Vision Transformer for Fine-Grained Visual Categorization

Bin similarity‐based domain adaptation for fine‐grained image classification

RAMS-Trans: Recurrent Attention Multi-scale Transformer forFine-grained Image Recognition

Contact Info

Product

Resources

About