Few-Shot Object Detection via Feature Reweighting

Abstract

Conventional training of a deep CNN based object detector demands a large number of bounding box annotations, which may be unavailable for rare categories. In this work we develop a few-shot object detector that can learn to detect novel objects from only a few annotated examples. Our proposed model leverages fully labeled base classes and quickly adapts to novel classes, using a meta feature learner and a reweighting module within a one-stage detection architecture. The feature learner extracts meta features that are generalizable to detect novel object classes, using training data from base classes with sufficient samples. The reweighting module transforms a few support examples from the novel classes to a global vector that indicates the importance or relevance of meta features for detecting the corresponding objects. These two modules, together with a detection prediction module, are trained end-to-end based on an episodic few-shot learning scheme and a carefully designed loss function. Through extensive experiments we demonstrate that our model outperforms well-established baselines by a large margin for few-shot object detection, on multiple datasets and settings. We also present analysis on various aspects of our proposed model, aiming to provide some inspiration for future few-shot detection works.

Meta R-CNN: Towards General Solver for Instance-Level Low-Shot Learning

“…The methods apply covnets to classify and regress the location by the region proposals generated by different algorithms [40,36]. More recently, low-shot object detection has been extended from recognition [4,22,21]. [21] follows fullimage meta-learning principle to address this problem.…”

“…More recently, low-shot object detection has been extended from recognition [4,22,21]. [21] follows fullimage meta-learning principle to address this problem. Instead, we discuss the similarity and difference between lowshot object recognition and detection in Sec 3, to reasonably motivate our RoI meta-learning approach.…”

“…Benchmarks and setups. Our low-shot object detection experiment follows the setup [21]. Concretely, we evaluate all baselines on the generic object detection tracks of PAS-CAL VOC 2007 [9], 2012 [9], and MS-COCO [27] benchmarks.…”

“…The union of 80k train images and a 35k subset of validation images (trainval35k) are used for training, and our evaluation is based on the remaining 5k val images (minival). Finally, we consider the cross-benchmark transfer setup of low-shot object detection from COCO to PAS-CAL [21], which leverages 60 base classes of COCO to learn knowledge representations and the evaluation is based on 20 novel classes of PASCAL VOC.…”

“…Finally, FRCN+ft-full employ the same training strategy of FRCN+ft in the first phase, yet train the detector to fully converge in the second phase. Beyond these baselines, Meta R-CNN is also compared with the state-of-the-art low-shot object detector [21] modified from YOLOv2 [35] (YOLO-Low-shot). Note that, YOLO-Low-shot also employs metalearning, whereas distinct from Meta R-CNN based on RoI features, it is based on a full image.…”