Continual BatchNorm Adaptation (CBNA) for Semantic Segmentation

Klingner, Marvin; Ayache, Mouadh; Fingscheidt, Tim

doi:10.1109/tits.2022.3190263

Cited by 11 publications

(7 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similar settings involving multiple target domains, which are hence closely related to classical CL, are also addressed for semantic segmentation. CBNA [16] mixes statistics from S and T to update the batch normalization layers and showcases the efficacy of the approach on continually visited target domains. CoTTA [33] adapts the entire network without using source data but self-supervision.…”

Section: Related Workmentioning

confidence: 99%

Continual SLAM: Beyond Lifelong Simultaneous Localization and Mapping Through Continual Learning

Vödisch

Cattaneo

Burgard

et al. 2023

Springer Proceedings in Advanced Robotics

View full text Add to dashboard Cite

Visual odometry is a fundamental task for many applications on mobile devices and robotic platforms. Since such applications are oftentimes not limited to predefined target domains and learning-based vision systems are known to generalize poorly to unseen environments, methods for continual adaptation during inference time are of significant interest. In this work, we introduce CoVIO for online continual learning of visual-inertial odometry. CoVIO effectively adapts to new domains while mitigating catastrophic forgetting by exploiting experience replay. In particular, we propose a novel sampling strategy to maximize image diversity in a fixed-size replay buffer that targets the limited storage capacity of embedded devices. We further provide an asynchronous version that decouples the odometry estimation from the network weight update step enabling continuous inference in real time. We extensively evaluate CoVIO on various real-world datasets demonstrating that it successfully adapts to new domains while outperforming previous methods. The code of our work is publicly available at http://continual-slam.cs.uni-freiburg.de.

show abstract

Section: Related Workmentioning

confidence: 99%

Continual SLAM: Beyond Lifelong Simultaneous Localization and Mapping Through Continual Learning

Vödisch

Cattaneo

Burgard

et al. 2023

Springer Proceedings in Advanced Robotics

View full text Add to dashboard Cite

show abstract

“…Similar settings involving multiple target domains, which are hence closely related to classical CL, are also addressed for semantic segmentation. CBNA [17] mixes statistics from S and T to update the batch normalization layers and showcases the efficacy of the approach on continually visited target domains. CoTTA [35] adapts the entire network without using source data but self-supervision.…”

Section: Related Workmentioning

confidence: 99%

CoDEPS: Online Continual Learning for Depth Estimation and Panoptic Segmentation

V�disch,

Petek,

Burgard

et al. 2023

Robotics: Science and Systems XIX

View full text Add to dashboard Cite

Operating a robot in the open world requires a high level of robustness with respect to previously unseen environments. Optimally, the robot is able to adapt by itself to new conditions without human supervision, e.g., automatically adjusting its perception system to changing lighting conditions. In this work, we address the task of continual learning for deep learning-based monocular depth estimation and panoptic segmentation in new environments in an online manner. We introduce CoDEPS to perform continual learning involving multiple real-world domains while mitigating catastrophic forgetting by leveraging experience replay. In particular, we propose a novel domain-mixing strategy to generate pseudo-labels to adapt panoptic segmentation. Furthermore, we explicitly address the limited storage capacity of robotic systems by leveraging sampling strategies for constructing a fixed-size replay buffer based on rare semantic class sampling and image diversity. We perform extensive evaluations of CoDEPS on various realworld datasets demonstrating that it successfully adapts to unseen environments without sacrificing performance on previous domains while achieving state-of-the-art results. The code of our work is publicly available at http://codeps.cs.uni-freiburg.de. * Equal contribution.

show abstract

“…In source-free UDA, in contrast, the adaptation process to the new domains must occur without forgetting essential information from the source domain. In this case, the only information that may be used from the source domain is the implicit information in the network weights from the pre-training on the source domain, which includes normalization parameters [24], [25].…”

Section: ) Source-free Udamentioning

confidence: 99%

Survey on Unsupervised Domain Adaptation for Semantic Segmentation for Visual Perception in Automated Driving

Schwonberg¹,

Niemeijer

Termöhlen

et al. 2023

IEEE Access

Self Cite

View full text Add to dashboard Cite

Deep neural networks (DNNs) have proven their capabilities in the past years and play a significant role in environment perception for the challenging application of automated driving. They are employed for tasks such as detection, semantic segmentation, and sensor fusion. Despite tremendous research efforts, several issues still need to be addressed that limit the applicability of DNNs in automated driving. The bad generalization of DNNs to unseen domains is a major problem on the way to a safe, large-scale application, because manual annotation of new domains is costly, particularly for semantic segmentation. For this reason, methods are required to adapt DNNs to new domains without labeling effort. This task is termed unsupervised domain adaptation (UDA). While several different domain shifts challenge DNNs, the shift between synthetic and real data is of particular importance for automated driving, as it allows the use of simulation environments for DNN training. We present an overview of the current state of the art in this research field. We categorize and explain the different approaches for UDA. The number of considered publications is larger than any other survey on this topic. We also go far beyond the description of the UDA state-of-the-art, as we present a quantitative comparison of approaches and point out the latest trends in this field. We conduct a critical analysis of the state-of-the-art and highlight promising future research directions. With this survey, we aim to facilitate UDA research further and encourage scientists to exploit novel research directions.

show abstract

Continual BatchNorm Adaptation (CBNA) for Semantic Segmentation

Cited by 11 publications

References 59 publications

Continual SLAM: Beyond Lifelong Simultaneous Localization and Mapping Through Continual Learning

Continual SLAM: Beyond Lifelong Simultaneous Localization and Mapping Through Continual Learning

CoDEPS: Online Continual Learning for Depth Estimation and Panoptic Segmentation

Survey on Unsupervised Domain Adaptation for Semantic Segmentation for Visual Perception in Automated Driving

Contact Info

Product

Resources

About