Attribution-aware Weight Transfer: A Warm-Start Initialization for Class-Incremental Semantic Segmentation

Ecosystem Health of the Beiyun River Basin (Beijing, China) as Evaluated by the Analytic Hierarchy Process Method

“…3, the first category, known as data-replay methods, involves storing a portion of past training data as exemplar memory such as [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36]. The second category, termed datafree methods, includes methods like [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48], [49]. These methods utilize transfer learning techniques, such as knowledge distillation, to inherit the capabilities of the old model.…”

“…Here we would like to discuss the advantage and necessity of continual learning based on specified models during the period of emerging large models. Although recent [26], [34], [35], [50], [51] Generative-replay Generative-data Generative-feature without storing real data, customized data replay heavy reliance on generative quality, high space complexity [28], [29], [32], [36], [52] Self-supervised Contrastive-learning Pseudo-labeling Foundation-model Driven strong adaptability, exemplar-memory free high training cost, hard to convergence [27], [41], [48], [53], [54] Regularization-based [39], [40], [43], [44], [55] Dynamic-architecture Parameter Allocation Architecture Decomposition Modular Network high model flexibility, high adaptability to diverse data network parameters gradually increases, high space complexity [30], [46], [56], [57], [58] large-model forms [59], [60] achieve fair zero-shot learning ability, they often lack the ability to classify targets with semantic understanding like humans. Another significant concern is cost.…”

“…Zhang et al [160] build an importance-based selective regularization method for inheritance from the old model. AWT [46] identifies the most relevant weights for new classes from the classifier's weights for the previous background and transfers these weights to the new classifier. GSC [108] attempts to alleviate the forgetting problem from the gradient aspect.…”

“…Moreover, since the model architecture keeps consistent, a solid weight transfer can effectively initialize the new model. On this route, ALIFE [32], AWT [56] and EWF [47] focus on weight transfer and parameter fusion to boost the classifier on new tasks or classes. Of which AWT [46] builds an attribution method to identify the most relevant classifier channels for predicting the new classes and transfer only those channel weights.…”