MS-KD: Multi-Organ Segmentation with Multiple Binary-Labeled Datasets

Abstract

Annotating multiple organs in 3D medical images is timeconsuming and costly. Meanwhile, there exist many single-organ datasets with one specific organ annotated. This paper investigates how to learn a multi-organ segmentation model leveraging a set of binary-labeled datasets. A novel Multi-teacher Single-student Knowledge Distillation (MS-KD) framework is proposed, where the teacher models are pretrained single-organ segmentation networks, and the student model is a multi-organ segmentation network. Considering that each teacher focuses on different organs, a region-based supervision method, consisting of logits-wise supervision and feature-wise supervision, is proposed. Each teacher supervises the student in two regions, the organ region where the teacher is considered as an expert and the back-ground region where all teachers agrees. Extensive experiments on three public single-organ datasets and a multi-organ dataset have demonstrated the effectiveness of the proposed MS-KD framework.

UFPS: A unified framework for partially annotated federated segmentation in heterogeneous data distribution

“…Cross pseudo supervision (CPS), 39 a pseudolabel-based segmentation method, uses Siamese networks supervising each other to correct potential noises in pseudolabels. Another way in this research field is MS-KD, 40 which pretrains teacher models based on several datasets, each for one organ, to generate pseudolabels. In multi-teacher single-student knowledge distillation (MS-KD), features from all layers are distilled along with final logits to ease model training using the Kullback-Leibler (KL) loss.…”

Learning multi‐organ and tumor segmentation from partially labeled datasets by a conditional dynamic attention network

“…Such a design is redundant and with poor flexibility. Recently, two other types of solutions have been proposed, that is, pseudo-label-based methods, [18][19][20][21] and conditional information-based methods. 22,23 Most pseudo-label-based approaches obtain teacher networks from single-organ datasets and then teach a student network by generating pseudo-labels.…”