Learning a Unified Classifier Incrementally via Rebalancing

Hou, Saihui; Pan, Xinyu; Loy, Chen Change; Wang, Zhong Lin; Lin, Dahua

doi:10.1109/cvpr.2019.00092

Cited by 875 publications

(1,096 citation statements)

References 18 publications

Supporting

Mentioning

1,091

Contrasting

Order By: Relevance

“…Javed et al [ 80 ] proposed a dynamic threshold shift method to improve the limitations of the deviation in a general knowledge distillation model. Hou et al [ 81 ] integrated cosine normalization, less-forget constraint and inter-class separation into a distillation model to mitigate the negative influences of the imbalance between new and old data.…”

Section: Methods Descriptionmentioning

confidence: 99%

An Appraisal of Incremental Learning Methods

Luo

Yin

Bai

et al. 2020

Entropy

View full text Add to dashboard Cite

As a special case of machine learning, incremental learning can acquire useful knowledge from incoming data continuously while it does not need to access the original data. It is expected to have the ability of memorization and it is regarded as one of the ultimate goals of artificial intelligence technology. However, incremental learning remains a long term challenge. Modern deep neural network models achieve outstanding performance on stationary data distributions with batch training. This restriction leads to catastrophic forgetting for incremental learning scenarios since the distribution of incoming data is unknown and has a highly different probability from the old data. Therefore, a model must be both plastic to acquire new knowledge and stable to consolidate existing knowledge. This review aims to draw a systematic review of the state of the art of incremental learning methods. Published reports are selected from Web of Science, IEEEXplore, and DBLP databases up to May 2020. Each paper is reviewed according to the types: architectural strategy, regularization strategy and rehearsal and pseudo-rehearsal strategy. We compare and discuss different methods. Moreover, the development trend and research focus are given. It is concluded that incremental learning is still a hot research area and will be for a long period. More attention should be paid to the exploration of both biological systems and computational models.

show abstract

Section: Methods Descriptionmentioning

confidence: 99%

An Appraisal of Incremental Learning Methods

Luo

Yin

Bai

et al. 2020

Entropy

View full text Add to dashboard Cite

show abstract

“…And the exemplar selection method in [15] needs to be improved. As shown in [16], the imbalance between old and new classes is a crucial reason for catastrophic forgetting, cosine normalization, and inter-class separation are used to address the problem. In [17], a linear model is added after the FC layer to alleviate the impact of class imbalance.…”

Section: B Incremental Learning With Deep Learningmentioning

confidence: 99%

“…The important network parameters are more robust in [5]- [8], and these works can be used for reference. But the main drawback of these methods is that they cannot be kept well in a long sequence of classes or tasks [16]. Based on the considerations above, memory-based methods, and model-based methods are combined in our work, and MAS is adopted for representation learning and classification.…”

Section: B Incremental Learning With Deep Learningmentioning

confidence: 99%

“…data of the old and new classes, which is more realistic, but more difficult to train [16]. In this paper, we focus on classincremental learning.…”

Section: Introductionmentioning

confidence: 99%

“…For model-based methods [3]- [12], they update the weight parameters by using specific learning algorithms or certain defined loss functions. However, these methods cannot work well in long sequences of new classes or tasks, due mainly to the omitting of as the old data [16].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Exemplar-Supported Representation for Effective Class-Incremental Learning

Guo

Xie

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Catastrophic forgetting is a key challenge for class-incremental learning with deep neural networks, where the performance decreases considerably while dealing with long sequences of new classes. To tackle this issue, in this paper, we propose a new exemplar-supported representation for incremental learning (ESRIL) approach that consists of three components. First, we use memory aware synapses (MAS) pre-trained on the ImageNet to retain the ability of robust representation learning and classification for old classes from the perspective of the model. Second, exemplar-based subspace clustering (ESC) is utilized to construct the exemplar set, which can keep the performance from various views of the data. Third, the nearest class multiple centroids (NCMC) is used as the classifier to save the training cost of the fully connected layer of MAS when the criterion is met. Intensive experiments and analyses are presented to show the influence of various backbone structures and the effectiveness of different components in our model. Experiments on several general-purpose and fine-grained image recognition datasets have fully demonstrated the efficacy of the proposed methodology. INDEX TERMS Exemplar-based subspace clustering, incremental learning, memory aware synapses, image recognition.

show abstract