Few-Shot Class-Incremental Learning from an Open-Set Perspective

Peng, Can; Zhao, Kun; Wang, Tianren; Li, Meng; Lovell, Brian C.

doi:10.1007/978-3-031-19806-9_22

Cited by 47 publications

(36 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The remaining sessions comprise a small number of classes and shots (typically 5/10-way 5-shot). Here we follow the exact FSCIL settings as described in [63,37] for CIFAR100 and CUB200. We use ResNet-18/20 and EfficientNet-B0 as backbones.…”

Section: Class-incremental Learning Comparisonsmentioning

confidence: 99%

“…Similarly, for FACT, we use the default values α = 0.5, γ = 0.01, V = number of new classes in total [63]. For ALICE, following [37], the projection head is a two-layer MLP with a hidden feature size of 2048 and ReLU as the activation function. All the other hyperparameters (scale factor s, margin m, etc.)…”

Section: Extra Training Detailsmentioning

confidence: 99%

“…All the other hyperparameters (scale factor s, margin m, etc.) are set as in [37]. For E-EWC+SDC, a triplet loss [16] is used as in [59] and the final embeddings of 640 dimensions are normalized.…”

Section: Extra Training Detailsmentioning

confidence: 99%

“…Current SOTA methods for class incremental learning start from a pre-trained backbone [25,34,52, 51] and then adapt the features at each session of continual learning. The use of a pre-trained backbone has been shown to lead to strong performance especially in the few-shot CIL setting due to the lack of data [37]. However, it is unclear to what extent continuous adaption of the features in each session is helpful.…”

Section: Introductionmentioning

confidence: 99%

“…We report the accuracy (%) (↑) of the last session and the PPDR (%) (↓) in parentheses. Asterisk (*) indicates that the reported results are from [63] and † indicates results reported in[37]. We use three different backbones, EfficientNet-B0 (EN-B0) and ResNet-18/20 (RN-18/20); EN-B0 and RN-18 are pre-trained on Imagenet-1k.…”

mentioning

confidence: 99%

See 4 more Smart Citations

First Session Adaptation: A Strong Replay-Free Baseline for Class-Incremental Learning

Panos¹,

Kobe²,

Reino³

et al. 2023

Preprint

View full text Add to dashboard Cite

In Class-Incremental Learning (CIL) an image classification system is exposed to new classes in each learning session and must be updated incrementally. Methods approaching this problem have updated both the classification head and the feature extractor body at each session of CIL. In this work, we develop a baseline method, First Session Adaptation (FSA), that sheds light on the efficacy of existing CIL approaches, and allows us to assess the relative performance contributions from head and body adaption. FSA adapts a pre-trained neural network body only on the first learning session and fixes it thereafter; a head based on linear discriminant analysis (LDA), is then placed on top of the adapted body, allowing exact updates through CIL. FSA is replay-free i.e. it does not memorize examples from previous sessions of continual learning. To empirically motivate FSA, we first consider a diverse selection of 22 imageclassification datasets, evaluating different heads and body adaptation techniques in high/low-shot offline settings. We find that the LDA head performs well and supports CIL outof-the-box. We also find that Featurewise Layer Modulation (FiLM) adapters are highly effective in the few-shot setting, and full-body adaption in the high-shot setting. Second, we empirically investigate various CIL settings including highshot CIL and few-shot CIL, including settings that have previously been used in the literature. We show that FSA significantly improves over the state-of-the-art in 15 of the 16 settings considered. FSA with FiLM adapters is especially performant in the few-shot setting. These results indicate that current approaches to continuous body adaptation are not working as expected. Finally, we propose a measure that can be applied to a set of unlabelled inputs which is predictive of the benefits of body adaptation.

show abstract

Section: Class-incremental Learning Comparisonsmentioning

confidence: 99%

Section: Extra Training Detailsmentioning

confidence: 99%