Multi-layer Domain Adaptation for Deep Convolutional Networks

Abstract

Despite their success in many computer vision tasks, convolutional networks tend to require large amounts of labeled data to achieve generalization. Furthermore, the performance is not guaranteed on a sample from an unseen domain at test time, if the network was not exposed to similar samples from that domain at training time. This hinders the adoption of these techniques in clinical setting where the imaging data is scarce, and where the intra-and inter-domain variance of the data can be substantial. We propose a domain adaptation technique that is especially suitable for deep networks to alleviate this requirement of labeled data. Our method utilizes gradient reversal layers [4] and Squeezeand-Excite modules [6] to stabilize the training in deep networks. The proposed method was applied to publicly available histopathology and chest X-ray databases and achieved superior performance to existing state-of-the-art networks with and without domain adaptation. Depending on the application, our method can improve multi-class classification accuracy by 5-20% compared to DANN introduced in [4].

Grasp-Oriented Fine-grained Cloth Segmentation without Real Supervision

“…As we show in the experimental section, training the U-Net network uniquely with synthetically generated data S s does not generalize well to real depth maps. In order to narrow the domain gap between real and synthetic depth maps, we introduce a multi-layer domain adaptation strategy [17]. More specifically, the features extracted at each encoder block of the U-Net, are forwarded to multiple classifiers, which aim at distinguishing between the feature maps belonging to real (X r i ) or synthetic (X s i ) samples.…”

Resource Efficient Domain Adaptation

“…Many novel methods have emerged recently [46,54,62,70]. For instance, Ciga et al [5] introduced domain discriminators at multiple layers of deep networks. However, we found that simply applying DA loss to multiple layers in multi-exit architectures [27] is not satisfactory.…”

Adversarial Normalization for Multi Domain Image Segmentation

“…By using λ = 0, the model becomes similar to (Drozdzal et al, 2018), where the generator is not constrained to produce realistic images. In contrast, for a large λ, our model becomes similar to an adversarial domain classifier (Ciga et al, 2019), where generated images are normalized across different domains but not optimal for segmentation, with added realism constraints.…”

“…For instance, Onofrey et al (2019) evaluate the benefit of using different normalization techniques to multi-site prostate MRI before applying deep learning-based segmentation. Recently, a few studies have explored the potential of learning methods for dynamic data augmentation and normalization (Drozdzal et al, 2018;Ciga et al, 2019;Hesse et al, 2020) as well as image denoising (Oguz et al, 2020). Drozdzal et al (2018) use two consecutive fully-convolutional CNNs, a pre-processor network followed by a segmentation network trained with a Dice metric, to normalize an input image prior to segmentation.…”

“…However, these standard normalization techniques are not learned for a specific task, certainly leading to suboptimal results compared to a task-driven normalization approach. Gradient reversal layer based domain adaptation and layer-wise domain-adversarial networks are used in [6] but this method is limited to two domains.…”