Multi-Domain Incremental Learning for Semantic Segmentation

Abstract

Recent efforts in multi-domain learning for semantic segmentation attempt to learn multiple geographical datasets in a universal, joint model. A simple fine-tuning experiment performed sequentially on three popular road scene segmentation datasets demonstrates that existing segmentation frameworks fail at incrementally learning on a series of visually disparate geographical domains. When learning a new domain, the model catastrophically forgets previously learned knowledge. In this work, we pose the problem of multi-domain incremental learning for semantic segmentation. Given a model trained on a particular geographical domain, the goal is to (i) incrementally learn a new geographical domain, (ii) while retaining performance on the old domain, (iii) given that the previous domain's dataset is not accessible. We propose a dynamic architecture that assigns universally shared, domain-invariant parameters to capture homogeneous semantic features present in all domains, while dedicated domain-specific parameters learn the statistics of each domain. Our novel optimization strategy helps achieve a good balance between retention of old knowledge (stability) and acquiring new knowledge (plasticity). We demonstrate the effectiveness of our proposed solution on domain incremental settings pertaining to realworld driving scenes from roads of Germany (Cityscapes), the United States (BDD100k), and India (IDD).

Causes of Catastrophic Forgetting in Class-Incremental Semantic Segmentation

“…Research on domainincremental learning for semantic segmentation is relatively sparse. Garg et al [19] propose a dynamic architecture that learns dedicated parameters to capture domain-specific features for each domain. Mirza et al [40] circumvent the issues of biased BatchNorm statistics by re-estimating and saving them for every domain, so that during inference domain-specific statistics can be used.…”

Generative appearance replay for continual unsupervised domain adaptation

“…We herein group them collectively under the umbrella of sequential UDA, and propose the taxonomy presented in Table 1 to clarify the differences among the individual settings. We denote the problem of learning multiple labeled target domains without retaining previously seen data as domain-incremental learning (DIL) [50], [51], [52], [53], [54], [55], [56], [57]. Another sub-category called source-free UDA -sometimes referred to as unsupervised model adaptation (UMA) -has recently gained increasing attention for both classification [58], [59], [60], [61] and segmentation tasks [62], [63], [64], [65], [66].…”

Semantic Image Segmentation: Two Decades of Research

“…For what concerns methods designed ad hoc for SiS, different works consider the problem of learning different domains over the lifespan of a model (Wu et al, 2019;Porav et al, 2019;Garg et al, 2022). They face the domain-incremental problem by assuming that data from new domains come unlabeled -and, therefore, they are more connected to the DASiS literature where the typical task is unsupervised DA Wu et al (2019) propose to generate data that resembles that of the current target domain and to update the model's parameters relying on such samples.…”

“…They build their method by using GANs, and the proposed approach does not require domain-specific finetuning. Instead, Garg et al (2022) learn domain-specific parameters for each new domain -in their case corresponding to different geographical regionswhereas other parameters are assumed to be domain-invariant.…”