Dual-Task Self-supervision for Cross-modality Domain Adaptation

Xue, Yingying; Feng, Shixiang; Zhang, Ya; Zhang, Xiaoyun; Wang, Yanfeng

doi:10.1007/978-3-030-59710-8_40

Cited by 19 publications

(9 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given data from multiple sources, Domain-Prediction-based methods find a harmonized data representation such that all information relating to the source domain of the image is removed [6,7,14]. This goal can be achieved by appending another head to the network, which acts as a domain classifier.…”

Section: Domain-prediction-based Methodsmentioning

confidence: 99%

How Reliable Are Out-of-Distribution Generalization Methods for Medical Image Segmentation?

Sanner¹,

González²,

Mukhopadhyay³

2021

Preprint

View full text Add to dashboard Cite

The recent achievements of Deep Learning rely on the test data being similar in distribution to the training data. In an ideal case, Deep Learning models would achieve Out-of-Distribution (OoD) Generalization, i.e. reliably make predictions on out-of-distribution data. Yet in practice, models usually fail to generalize well when facing a shift in distribution. Several methods were thereby designed to improve the robustness of the features learned by a model through Regularizationor Domain-Prediction-based schemes. Segmenting medical images such as MRIs of the hippocampus is essential for the diagnosis and treatment of neuropsychiatric disorders. But these brain images often suffer from distribution shift due to the patient's age and various pathologies affecting the shape of the organ. In this work, we evaluate OoD Generalization solutions for the problem of hippocampus segmentation in MR data using both fully-and semi-supervised training. We find that no method performs reliably in all experiments. Only the V-REx loss stands out as it remains easy to tune, while it outperforms a standard U-Net in most cases.

show abstract

Section: Domain-prediction-based Methodsmentioning

confidence: 99%

How Reliable Are Out-of-Distribution Generalization Methods for Medical Image Segmentation?

Sanner¹,

González²,

Mukhopadhyay³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The work in Ref. 49 showed that a hierarchical domain adaptation structure was very effective to adapt a U-shape model trained in one domain to another for segmentation. Inspired by their work, we develop a hierarchical adversarial learning structure for HRNet to compensate domain shift between our Datasets, T and B, for OCT speckle noise compression.…”

Section: Hierarchical Adversarial Learningmentioning

confidence: 99%

DHNet: High‐resolution and hierarchical network for cross‐domain OCT speckle noise reduction

Zhou

Wang

et al. 2022

Medical Physics

View full text Add to dashboard Cite

Purpose: Optical coherence tomography (OCT) imaging uses the principle of Michelson interferometry to obtain high-resolution images by coherent superposing of multiple forward and backward scattered light waves with random phases. This process inevitably produces speckle noise that severely compromises visual quality of OCT images and degrades performances of subsequent image analysis tasks. In addition, datasets obtained by different OCT scanners have distribution shifts, making a speckle noise suppression model difficult to be generalized across multiple datasets. In order to solve the above issues, we propose a novel end-to-end denoising framework for OCT images collected by different scanners. Methods: The proposed model utilizes the high-resolution network (HRNet) as backbone for image restoration, which reconstructs high-fidelity images by maintaining high-resolution representations throughout the entire learning process. To compensate distribution shifts among datasets collected by different scanners, we develop a hierarchical adversarial learning strategy for domain adaption. The proposed model is trained using datasets with clean ground truth produced by two commercial OCT scanners, and then applied to suppress speckle noise in OCT images collected by our recently developed OCT scanner, BV-1000 (China Bigvision Corporation). We name the proposed model as DHNet (Double-H-Net, High-resolution and Hierarchical Network). Results: We compare DHNet with state-of -the-art methods and experiment results show that DHNet improves signal-to-noise ratio by a large margin of 18.137 dB as compared to the best of our previous method. In addition, DHNet achieves a testing time of 25 ms, which satisfies the real-time processing requirement for the BV-1000 scanner. We also conduct retinal layer segmentation experiment on OCT images before and after denoising and show that DHNet can also improve segmentation. Conclusions:The proposed DHNet can compensate domain shifts between different data sets while significantly improve speckle noise suppression. The HRNet backbone is utilized to carry low-and high-resolution information to recover fidelity images. Domain adaptation is achieved by a hierarchical module through adversarial learning. In addition, DHNet achieved a testing time of 25 ms, which satisfied the real-time processing requirement.

show abstract

“…In domain adaptation approaches for medical image segmentation, a DL-based model trained with labeled dataset in a specific domain (e.g. normal CXR) is refined for different domain dataset in semi-supervised, self-supervised or unsupervised manners (Bai et al, 2017;Tang et al, 2019;Tarvainen and Valpola, 2017;Li et al, 2020a;Perone et al, 2019;Xue et al, 2020;Orbes-Arteaga et al, 2019;Li et al, 2020b;Chen et al, 2020). These approaches try to take advantages of learned features from the supervised learning in a specific domain, and distillate the knowledge to unsupervised learning tasks in unseen domains.…”

Section: Semi-supervised Learning Via Domain Adaptationmentioning

confidence: 99%

“…Recent emerging researches on self-supervised learning has brought large improvements in medical domain adaptation or image segmentation tasks, by promoting consistency between model outputs given a same input with different perturbations or by training an auxiliary proxy task (Xue et al, 2020;Orbes-Arteaga et al, 2019;Li et al, 2020b). In general, DL models trained with auxiliary self-supervised losses are proven to achieves better generalization capability as well as better primary task performance, especially when training with limited labeled dataset and abundant unlabeled dataset.…”

Section: Self-supervised Learningmentioning

confidence: 99%

“…For examples, (Xue et al, 2020) achieved SOTA performance on cross-modality domain adaptation aided by selfsupervised learning, by constructing an auxiliary edge generation task to leverage unlabeled domain dataset. (Orbes-Arteaga et al, 2019) and (Li et al, 2020b) introduced a consistency loss between outputs between a student and a teacher model, given unlabeled input with different augmentation.…”

Section: Self-supervised Learningmentioning

confidence: 99%

See 1 more Smart Citation

CXR Segmentation by AdaIN-based Domain Adaptation and Knowledge Distillation

Oh¹,

Ye²

2021

Preprint

View full text Add to dashboard Cite

As the segmentation labels are scarce, extensive researches have been conducted to train segmentation networks without labels or with only limited labels. In particular, semi-supervised domain adaptation and self-supervised learning have been introduced to distill knowledge from various tasks to improve the segmentation performance. However, these approaches appear different from each other, so it is not clear how these seemingly different approaches can be combined for better performance. Inspired by the recent StarGANv2 for multi-domain image translation, here we propose a novel segmentation framework via AdaIN-based knowledge distillation, where a single generator with AdaIN layers is trained along with the AdaIN code generator and style encoder so that the generator can perform both domain adaptation and segmentation. Specifically, our framework is designed to deal with difficult situations in chest X-ray (CXR) segmentation tasks where segmentation masks are only available for normal CXR data, but the trained model should be applied for both normal and abnormal CXR images. Since a single generator is used for abnormal to normal domain conversion and segmentation by simply changing AdaIN codes, the generator can synergistically learn the common features to improve segmentation performance. Experimental results using CXR data confirm that the trained network can achieve the state-of-the art segmentation performance for both normal and abnormal CXR images.

show abstract

Dual-Task Self-supervision for Cross-modality Domain Adaptation

Cited by 19 publications

References 16 publications

How Reliable Are Out-of-Distribution Generalization Methods for Medical Image Segmentation?

How Reliable Are Out-of-Distribution Generalization Methods for Medical Image Segmentation?

DHNet: High‐resolution and hierarchical network for cross‐domain OCT speckle noise reduction

CXR Segmentation by AdaIN-based Domain Adaptation and Knowledge Distillation

Contact Info

Product

Resources

About