Unsupervised BatchNorm Adaptation (UBNA): A Domain Adaptation Method for Semantic Segmentation Without Using Source Domain Representations

Klingner, Marvin; Termöhlen, Jan-Aike; Ritterbach, Jacob; Fingscheidt, Tim

doi:10.1109/wacvw54805.2022.00027

Cited by 34 publications

(32 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is disadvantageous for continual UDA settings, where it would be desirable to continuously adapt on a single-image basis to avoid algorithmic delay during deployment in rapidly changing domains. In contrast to existing methods [19], [22], [23], [24], [15], our CBNA method is applicable to these continual UDA settings, which we will show by our successful single-image adaptation results without the usage of additional uncorrelated images. Another novelty of CBNA is its integration into the single-image inference forward pass of an already trained model, which introduces nearly no computational overhead during inference.…”

Section: Uda Via Normalization Layersmentioning

confidence: 89%

“…The initial works of Li et al [22], [23] for image classification and Zhang et al [15] for semantic segmentation show that the re-estimation of batch normalization (BN) statistics in the target domain can be used for UDA without source data. The UBNA method from Klingner et al [19] has shown that mixing statistics from the source and target domain outperforms these initial works, which we build upon for our method design. These findings for domain adaptation are also supported by the work on adversarial robustness of Schneider et al [24], where the beneficial effect of mixing statistics from perturbed and clean images is shown.…”

Section: Uda Via Normalization Layersmentioning

confidence: 99%

“…2. Overview on how our novel CBNA approach differs from other source-free UDA approaches, e.g., UBNA [19]. Re-initialization and adaptation with CBNA is performed for each new image during inference.…”

Section: Introductionmentioning

confidence: 99%

“…of the source domain and a single target domain image in a continual fashion for each new image from the target domain. Thereby, during inference in a vehicle, we can adapt the model instantaneously to each new image from the video stream of a camera, while previous approaches [19], [22], [23], [24], [15] adapt only once in an offline fashion to a single target domain using multiple uncorrelated images. Regarding computational complexity, CBNA introduces only little computational overhead on the forward pass during inference, while reference approaches reaching a similar performance [19], [24] would require a whole additional forward pass through the model.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Continual BatchNorm Adaptation (CBNA) for Semantic Segmentation

Klingner¹,

Ayache²,

Fingscheidt³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Environment perception in autonomous driving vehicles often heavily relies on deep neural networks (DNNs), which are subject to domain shifts, leading to a significantly decreased performance during DNN deployment. Usually, this problem is addressed by unsupervised domain adaptation (UDA) approaches trained either simultaneously on source and target domain datasets or even source-free only on target data in an offline fashion. In this work, we further expand a source-free UDA approach to a continual and therefore online-capable UDA on a single-image basis for semantic segmentation. Accordingly, our method only requires the pre-trained model from the supplier (trained in the source domain) and the current (unlabeled target domain) camera image. Our method Continual BatchNorm Adaptation (CBNA) modifies the source domain statistics in the batch normalization layers, using target domain images in an unsupervised fashion, which yields consistent performance improvements during inference. Thereby, in contrast to existing works, our approach can be applied to improve a DNN continuously on a single-image basis during deployment without access to source data, without algorithmic delay, and nearly without computational overhead. We show the consistent effectiveness of our method across a wide variety of source/target domain settings for semantic segmentation. As part of this work, our code will be made publicly available. 1

show abstract

Section: Uda Via Normalization Layersmentioning

confidence: 89%

Section: Uda Via Normalization Layersmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Continual BatchNorm Adaptation (CBNA) for Semantic Segmentation

Klingner¹,

Ayache²,

Fingscheidt³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Subsequent works adapted this concept and introduced improvements such as dilated convolutions [7,62], atrous spatial pyramid pooling [9], state-of-the-art feature extractors [8,45], label relaxation [68], or high resolution networks [57]. Other approaches explore unsupervised domain adaptation techniques [32,27,58,61] to mitigate the domain shift problem for semantic segmentation and thereby facilitate the use of readily available synthetic datasets [50,53]. Moreover, computational complexity can be reduced tremendously with only marginal loss in performance due to the use of efficient architectures.…”

Section: Related Workmentioning

confidence: 99%

Improving Online Performance Prediction for Semantic Segmentation

Klingner¹,

Bär²,

Mross³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

In this work we address the task of observing the performance of a semantic segmentation deep neural network (DNN) during online operation, i.e., during inference, which is of high importance in safety-critical applications such as autonomous driving. Here, many high-level decisions rely on such DNNs, which are usually evaluated offline, while their performance in online operation remains unknown. To solve this problem, we propose an improved online performance prediction scheme, building on a recently proposed concept of predicting the primary semantic segmentation task's performance. This can be achieved by evaluating the auxiliary task of monocular depth estimation with a measurement supplied by a LiDAR sensor and a subsequent regression to the semantic segmentation performance. In particular, we propose (i) sequential training methods for both tasks in a multi-task training setup, (ii) to share the encoder as well as parts of the decoder between both task's networks for improved efficiency, and (iii) a temporal statistics aggregation method, which significantly reduces the performance prediction error at the cost of a small algorithmic latency. Evaluation on the KITTI dataset shows that all three aspects improve the performance prediction compared to previous approaches.

show abstract