SS-IL: Separated Softmax for Incremental Learning

Ahn, Hongjoon; Kwak, Jihwan; Lim, Subin; Bang, Hyeonsu; Kim, Hyojun; Moon, Taesup

doi:10.1109/iccv48922.2021.00088

Cited by 120 publications

(71 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This section shows comparison experiments to illustrate the effectiveness of our GLFC model, as shown in Tables 1, 2, 3, where ' ' denotes the improvements of our model compared with other comparison methods. We observe that our model outperforms existing class-incremental methods [1,11,17,37,43,49] in the FCIL setting by a margin of 4.4%∼15.1% in terms of average accuracy. It validates that our model could enable local clients to collaboratively train a global class-incremental model.…”

Section: Performance Comparisonmentioning

confidence: 74%

“…Class-Incremental Learning (CIL) aims to learn new classes continuously while tackling forgetting on old classes [1,19,54]. Without access to the data of old classes, [20] designs new regulators for balancing the biased model optimization caused by new classes, and [25,41] use the knowledge distillation to surmount catastrophic forgetting.…”

Section: Related Workmentioning

confidence: 99%

“…[43] introduces geodesic path to traditional knowledge distillation. [1] combines task-wise knowledge distillation and separated softmax for bias compensation. These CIL methods however cannot be applied to tackle our FCIL problem, due to their strong assumptions on when and where the data of new classes arrive.…”

Section: Related Workmentioning

confidence: 99%

“…These probabilities reflect the interclass relations between old and new classes. Different from existing knowledge distillation strategies [1,3,8,17,26] that only ensure old classes' semantic consistency among Θ t−1 l and Θ r,t l , we consider inter-class relations between old and new classes simultaneously via optimizing L RD . Namely, we utilize a variant of one-hot encoding labels…”

Section: Local Catastrophic Forgetting Compensationmentioning

confidence: 99%

“…We use three datasets: CIFAR-100 [13,21], ImageNet-Subset [7], and TinyImageNet [35] in our experiments. For a fair comparison with baseline class-incremental learning methods [1,11,17,37,43,49] in the FCIL setting, we follow the same protocols proposed by [37,49] to set incremental tasks, utilize the identical class order generated from iCaRL [37], and employ the same backbone (i.e., ResNet-18 [15]) as the classification model [2]. The SGD optimizer whose learning rate is 2.0 is used to train all models.…”

Section: Implementation Detailsmentioning

confidence: 99%

See 4 more Smart Citations

Federated Class-Incremental Learning

Dong¹,

Wang²,

Fang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Federated learning (FL) has attracted growing attentions via data-private collaborative training on decentralized clients. However, most existing methods unrealistically assume object classes of the overall framework are fixed over time. It makes the global model suffer from significant catastrophic forgetting on old classes in real-world scenarios, where local clients often collect new classes continuously and have very limited storage memory to store old classes. Moreover, new clients with unseen new classes may participate in the FL training, further aggravating the catastrophic forgetting of global model. To address these challenges, we develop a novel Global-Local Forgetting Compensation (GLFC) model, to learn a global classincremental model for alleviating the catastrophic forgetting from both local and global perspectives. Specifically, to address local forgetting caused by class imbalance at the local clients, we design a class-aware gradient compensation loss and a class-semantic relation distillation loss to balance the forgetting of old classes and distill consistent inter-class relations across tasks. To tackle the global forgetting brought by the non-i.i.d class imbalance across clients, we propose a proxy server that selects the best old global model to assist the local relation distillation. Moreover, a prototype gradient-based communication mechanism is developed to protect the privacy. Our model outperforms state-of-the-art methods by 4.4%∼15.1% in terms of average accuracy on representative benchmark datasets. The code is available at https://github. com/conditionWang/FCIL.

show abstract

Section: Performance Comparisonmentioning

confidence: 74%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Local Catastrophic Forgetting Compensationmentioning

confidence: 99%

Section: Implementation Detailsmentioning

confidence: 99%

See 3 more Smart Citations

Federated Class-Incremental Learning

Dong¹,

Wang²,

Fang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

Data‐Driven Design of Electrically Conductive Nanocomposite Materials: A Case Study of Acrylonitrile–Butadiene–Styrene/Carbon Nanotube Binary Composites

Kim

Park

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

The field of polymer-based nanoscience has always been of significant interest in the search for polymer/carbon nanotube (CNT) nanocomposites with optimized material properties for new applications. Herein, it is demonstrated that data collected from the online literature can be used to develop an efficient deep learning model to design acrylonitrile-butadiene-styrene (ABS)/CNT binary composites. A dataset of 14 945 data points is constructed from 110 studies. The results demonstrate that compared with a vanilla deep regression model, the proposed model achieves a 26% lower average mean absolute error and an accuracy of 80.6%, which is 16.2% higher than the vanilla deep regression model in predicting the six electrical and mechanical properties of target ABS/CNT composites. In addition, a Monte Carlo simulation integrated with the developed deep neural network (DNN) model effectively screens input variables for users and thus appropriately guides them to manufacture a composite product with desired physical properties.

show abstract