PNPDet: Efficient Few-shot Detection without Forgetting via Plug-and-Play Sub-networks

Abstract

The human visual system can detect objects of unseen categories from merely a few examples. However, such capability remains absent in state-of-the-art detectors. To bridge this gap, several attempts have been proposed to perform few-shot detection by incorporating meta-learning techniques. Such methods can improve detection performance on unseen categories, but also add huge computational burden, and usually degrade detection performance on seen categories. In this paper, we present PNPDet, a novel Plug-and-Play Detector, for efficient few-shot detection without forgetting. It introduces a simple but effective architecture with separate sub-networks that disentangles the recognition of base and novel categories and prevents hurting performance on known categories while learning new concepts. Distance metric learning is further incorporated into sub-networks, consistently boosting detection performance for both base and novel categories. Experiments show that the proposed PNPDet can achieve comparable few-shot detection performance on unseen categories while not losing accuracy on seen categories, and also remain efficient and flexible at the same time.

A Survey of Deep Learning for Low-Shot Object Detection

“…NP-RepMet additionally defines a set of negative modes in the same form as positive modes for each class and proposes another loss function for negative modes. PNPDet [89] adopts a single-stage detection framework CenterNet. This methods first disentangles the recognition of base classes and novel classes by adding a parallel heatmap prediction sub-network for novel classes.…”

Defect-GAN: High-Fidelity Defect Synthesis for Automated Defect Inspection

“…Few-shot learning [51,52,12,5,31,22,62,57,70,10,55,72] refers to learning from extremely limited training samples (e.g., 1 or 3) for an unseen class. However, their performances are quite limited and thus far from practical application.…”

Attention Guided Cosine Margin For Overcoming Class-Imbalance in Few-Shot Road Object Detection

“…FSCE [28] adopts a contrastive training strategy while SRR-FSD [40] uses the semantic relationships between word embeddings from category labels to show improvements on novel class performance. PNPDet [37] decouples the base and novel class predictors and learns a cosine-similarity classifier that partially resolves catastrophic forgetting and class confusion. Unfortunately, metric learners suffer from extreme catastrophic forgetting as they tend to overfit on the novel classes.…”

“…Early attempts in FSOD have been made by drawing inspiration from two primary learning strategies in image classification -Meta-Learning [12], [15], [34], [36] and Metric Learning [28], [32], [37]. Benchmark experiments conducted * Work done as an intern at Intel.…”