Towards Generalized and Incremental Few-Shot Object Detection

Abstract

Real-world object detection is highly desired to be equipped with the learning expandability that can enlarge its detection classes incrementally. Moreover, such learning from only few annotated training samples further adds the flexibility for the object detector, which is highly expected in many applications such as autonomous driving, robotics, etc. However, such sequential learning scenario with few-shot training samples generally causes catastrophic forgetting and dramatic overfitting. In this paper, to address the above incremental few-shot learning issues, a novel Incremental Few-Shot Object Detection (iFSOD) method is proposed to enable the effective continual learning from few-shot samples. Specifically, a Double-Branch Framework (DBF) is proposed to decouple the feature representation of base and novel (few-shot) class, which facilitates both the old-knowledge retention and new-class adaption simultaneously. Furthermore, a progressive model updating rule is carried out to preserve the long-term memory on old classes effectively when adapt to sequential new classes. Moreover, an inter-task class separation loss is proposed to extend the decision region of new-coming classes for better feature discrimination. We conduct experiments on both Pascal VOC and MS-COCO, which demonstrate that our method can effectively solve the problem of incremental few-shot detection and significantly improve the detection accuracy on both base and novel classes.

Incremental Few-Shot Object Detection via Simple Fine-Tuning Approach

“…Better performance is needed for its application in a practical scenario. Fine-tuning-based works [21], [22] have been presented to achieve more accurate performance. They leverage knowledge distillation loss to overcome catastrophic forgetting.…”

“…2) Class-agnostic box regressor: Previous iFSD methods [15], [6], [22], [21] applied a class-specific box regressor to the object detector. A class-specific box regressor can generate a bounding box fitted to each class.…”

“…iMTFA [31] is designed for incremental few-shot semantic segmentation based on Mask R-CNN [4]; it creates novel classifier weights through weight imprinting. DBF [22], and LEAST [21] use Faster R-CNN as fine-tuning-based iFSD methods. DBF presents a progressive model updating rule to preserve the long-term memory of old classes.…”

CAM/CAD Point Cloud Part Segmentation via Few-Shot Learning

“…The major limitation in earlier more traditional methods (for recognition of 3D geometric features) is that the 3D characteristic of the work-pieces cannot be fully enumerated and considered in the automatic algorithms. With the recent advances in computer vision [8], [9] and applications of deep learning [10], the possibility for new developments now arises where the task of 3D feature detection can now be innovatively developed and formulated as a vision task involving the notion of semantic segmentation. Recently too, it is well-known that deep learning can be successfully adopted to address the semantic segmentation problem in both 2D and 3D [6], [12].…”