Meta R-CNN: Towards General Solver for Instance-Level Low-Shot Learning

Abstract

Resembling the rapid learning capability of human, lowshot learning empowers vision systems to understand new concepts by training with few samples. Leading approaches derived from meta-learning on images with a single visual object. Obfuscated by a complex background and multiple objects in one image, they are hard to promote the research of low-shot object detection/segmentation. In this work, we present a flexible and general methodology to achieve these tasks. Our work extends Faster /Mask R-CNN by proposing meta-learning over RoI (Region-of-Interest) features instead of a full image feature. This simple spirit disentangles multi-object information merged with the background, without bells and whistles, enabling Faster /Mask R-CNN turn into a meta-learner to achieve the tasks. Specifically, we introduce a Predictor-head Remodeling Network (PRN) that shares its main backbone with Faster /Mask R-CNN. PRN receives images containing low-shot objects with their bounding boxes or masks to infer their class attentive vectors. The vectors take channel-wise soft-attention on RoI features, remodeling those R-CNN predictor heads to detect or segment the objects that are consistent with the classes these vectors represent. In our experiments, Meta R-CNN yields the stateof theart inlow-shot object detection and improves low-shot object segmentation by Mask R-CNN.Code: https://yanxp.github.io/metarcnn.html. * indicate equal contribution (Xiaopeng Yan and Ziliang Chen). † indicates corresponding author: Liang Lin.

Few-Shot Object Detection via Sample Processing

“…[35] leveraged fully labeled base classes and quickly adapted them to novel classes using a metafeature learner and a reweighting module within a one-stage detection architecture. Analogously, [42] reweighted ROI features in the detection head with channelwise attention based on a twostage framework. Recent work [51] proposes a prototypical knowledge transfer with an attached meta-learner as well.…”

“…Pascal VOC annotates more than 30k images from 20 categories, and MS COCO has a more diverse set of 80 categories with 200k images, containing all 20 categories of Pascal VOC. To ensure the reliability of the experiment, we adopt the same settings as other SOTA FSOD detectors [35,41,42] to construct a fewshot detection dataset. We carried out three groups of experiments, i.e., experiments over the Pascal VOC dataset, experiments over the MS COCO dataset and experiments over the cross-benchmark set from MS COCO to PASCAL VOC.…”

“…To determine the performance of our FSOD detector, we conduct a set of contrast experiments with other state-of-theart FSOD detection counterparts. Specifically, LSTD [38], FSRW [35], Meta-RCNN [42], MetDet [41], FSOD-KT [51], FSOD-AM [40] and TFA [14] are compared with our model. In addition, YOLOv3-SPP is also used as the detector to predict the novel classes directly, which proves the effectiveness of our proposed method by comparison of the results.…”

Visual Analytics for Operation-Level Construction Monitoring and Documentation: State-of-the-Art Technologies, Research Challenges, and Future Directions

“…As active learning algorithms select the most informative-to-learn instances from abundant unlabeled training data and train a deep learning model with the selected data first, it is possible to significantly reduce the amount of training data and the human effort required for DB development (Kim et al, 2020b). Other researchers also investigated the few-shot learning algorithm (Kang et al, 2019;Wang et al, 2019c;Yan et al, 2019a), which aims to learn and detect new types of target objects even if a small amount of training data are given, i.e., less than 30 images. This would be very beneficial for construction sites where diverse types of construction resources exist and they often vary from phase to phase.…”

GenDet: Meta Learning to Generate Detectors From Few Shots

“…Pioneers of few-shot detection have explored background depression [15], metric learning [16] and feature reweighting [17], [18]. Most of these previous works carry out few-shot detection implicitly, either by manipulating the feature maps to highlight regions [15] or channels [17], [18] related to novel classes, or by constraining different classes to be separable in a learned embedding space [16]. Besides, [19] proposes a simple finetuning baseline for few-shot detection, introducing the instance-level feature normalization when finetuning on novel classes.…”