Target and Task specific Source-Free Domain Adaptive Image Segmentation

Abstract

Solving the domain shift problem during inference is essential in medical imaging as most deep-learning based solutions suffer from it. In practice, domain shifts are tackled by performing Unsupervised Domain Adaptation (UDA), where a model is adapted to an unlabeled target domain by leveraging the labelled source domain. In medical scenarios, the data comes with huge privacy concerns making it difficult to apply standard UDA techniques. Hence, a closer clinical setting is Source-Free UDA (SFUDA), where we have access to source trained model but not the source data during adaptation. Methods trying to solve SFUDA typically address the domain shift using pseudo-label based self-training techniques. However due to domain shift, these pseudo-labels are usually of high entropy and denoising them still does not make them perfect labels to supervise the model. Therefore, adapting the source model with noisy pseudo labels reduces its segmentation capability while addressing the domain shift. To this end, we propose a two-stage approach for source-free domain adaptive image segmentation: 1) Target-specific adaptation followed by 2) Task-specific adaptation. In the first stage, we focus on generating target-specific pseudo labels while suppressing high entropy regions by proposing an Ensemble Entropy Minimization loss. We also introduce a selective voting strategy to enhance pseudo-label generation. In the second stage, we focus on adapting the network for task-specific representation by using a teacher-student self-training approach based on augmentation-guided consistency. We evaluate our proposed method on both 2D fundus datasets and 3D MRI volumes across 7 different domain shifts where we achieve better performance than recent UDA and SF-UDA methods for medical image segmentation. Code is available at https://github.com/Vibashan/tt-sfuda

Source-free unsupervised adaptive segmentation for knee joint MRI

“…Both have been demonstrated for use in knee tissue segmentation [18,28]. For source-free UDA methods, we compared the proposed framework against DPL [22], TT-SFUDA [23], and SFUDA [24]. We excluded the method proposed by Bateson [21] because it introduces additional prior knowledge, which was unavailable in our experiments.…”

“…The number of noisy pseudo labels produced by directly feeding the target data into the source model was reduced, and the reliable labels were selected to guide the target model to reduce the domain gap using two complementary pixel-level (uncertainty estimation) and class-level (prototype estimation) denoising methods. Vibashan et al [23] developed a source-free UDA approach by introducing a selective voting method based on entropy maps to enhance pseudo labels for target-specific adaptation. The target model additionally uses a teacherstudent self-learning framework with augmentation-guided consistency for task-specific adaptation.…”

“…It has been reported that source models tend to have low prediction certainty (high entropy) for target predictions and high prediction certainty (low entropy) for source predictions [42]. Hence, in the sourcefree adaptation setting, minimizing the predictive entropy over unlabeled target images is generally utilized to guide targetspecific models to achieve more accurate predictions [23,24,43]. To improve the model's robustness, instead of adopting simple EM (as most works do), we introduced AEM to reduce the entropy of the augmented network output.…”

“…As ground truth labels of the target domain are unavailable, the pseudo labels generated in Stage I are used to supervise the target-specific model in Stage II. Unfortunately, gradually accumulated pseudo label errors may decrease the segmentation performance of the target images [23]. To improve the quality of the noisy pseudo labels, we applied a voting strategy to select target regions of pseudo labels to reduce false-negative pixels that were wrongly detected as background.…”

“…To overcome this problem, source-free UDA is proposed for model adaptation because it requires only a well-trained source model and unlabeled target data [19,20]. A few source-free UDA methods have been explored in the medical domain but have not been applied for the segmentation of knee joint tissue [21][22][23][24]. Consequently, there is a practical need for a reliable and robust source-free UDA segmentation network to achieve knee joint segmentation in unlabeled MR images that are acquired with different acquisition settings without access to the source data.…”