Query Adaptive Few-Shot Object Detection with Heterogeneous Graph Convolutional Networks

Han, Guangxing; He, Yicheng; Huang, Shiyuan; Ma, Jiawei; Chang, Shih-Fu

doi:10.1109/iccv48922.2021.00325

Cited by 87 publications

(28 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Han et al [13] propose to perform feature alignment between the two inputs and focus on foreground regions using attention. GCNs are employed in [12] to facilitate mutual adaptation between the two branches. Other works [2,3,6,20] use more advanced nonlocal attention/transformer [40,44] to improve the similarity learning of the two inputs.…”

Section: Related Workmentioning

confidence: 99%

“…Our work belongs to the two-branch based few-shot object detection method. The motivation is that although the traditional twobranch based methods [8,12,13,20,23,49] show promising results, the interaction of the query and support branch is only restricted in the detection head, while leaving hundreds of layers for separate feature extraction in each branch before the cross-branch interaction. Our idea is to remove the separate deep feature encoders and fully exploit the cross-branch interaction to the largest extend.…”

Section: Overview Of Our Proposed Model (Fct)mentioning

confidence: 99%

“…We follow the vallina Faster R-CNN object detection framework, and do not use FPN [26] in our model. We find that using FPN does not improve the performance, especially for the two-branch based FSOD methods [3,8,12,20,48,49]. The cross-transformer based RoI feature extractor in the detection head can encourage mutual alignment between the query proposals and support images, which is crucial for the final pairwise matching.…”

Section: The Cross-transformer Detection Headmentioning

confidence: 99%

“…Current methods for this task mainly follow a twostage learning paradigm [45] to transfer the knowledge learned from the data-abundant base classes to assist in object detection for few-shot novel classes. The detailed model architectures vary in different works, which can be roughly divided into two categories, single-branch based methods [36,45,47,51,52] and two-branch based methods [8,12,13,20,23,49]. (1) Single-branch based methods employ a typical object detection model, e.g., Faster R-CNN [33], and build a multi-class classifier for detection.…”

Section: Introductionmentioning

confidence: 99%

“…(2) Two-branch based methods apply the metric-learning idea [34,37,41] to FSOD and build a siamese network to process the query image and the few-shot support image in parallel. After extracting deep visual features from the two branches, previous works propose various methods (e.g., feature fusion [8,48,49], feature alignment [13], GCN [12], and non-local attention/transformer [2,3,6,20,44]) to calculate the similarity of the two branches. The two-branch based methods do not learn the multi-class classifier over novel classes, and usually have stronger generalization ability by learning to compare the query regions with few-shot classes.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Few-Shot Object Detection with Fully Cross-Transformer

Han¹,

Ma²,

Huang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Few-shot object detection (FSOD), with the aim to detect novel objects using very few training examples, has recently attracted great research interest in the community. Metriclearning based methods have been demonstrated to be effective for this task using a two-branch based siamese network, and calculate the similarity between image regions and fewshot examples for detection. However, in previous works, the interaction between the two branches is only restricted in the detection head, while leaving the remaining hundreds of layers for separate feature extraction. Inspired by the recent work on vision transformers and vision-language transformers, we propose a novel Fully Cross-Transformer based model (FCT) for FSOD by incorporating cross-transformer into both the feature backbone and detection head. The asymmetric-batched cross-attention is proposed to aggregate the key information from the two branches with different batch sizes. Our model can improve the few-shot similarity learning between the two branches by introducing the multilevel interactions. Comprehensive experiments on both PAS-CAL VOC and MSCOCO FSOD benchmarks demonstrate the effectiveness of our model.

show abstract

Section: Related Workmentioning

confidence: 99%