MSKD: Structured knowledge distillation for efficient medical image segmentation

Zhao, Lichao; Qian, Xiaolong; Guo, Yinghui; Song, Jiaqi; Hou, Jinbao; Gong, Jun

doi:10.1016/j.compbiomed.2023.107284

Cited by 10 publications

(1 citation statement)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, a significant issue is the sharply increased computation in MGBS, which presents a challenge to the computational resources. In the future, we would like to explore strategies such as decreasing the batch size, implementing distributed training approaches (Langer et al 2020), and utilizing knowledge distillation techniques (Zhao et al 2023) to alleviate or address the computational challenges.…”

Section: Limitations and Future Directionmentioning

confidence: 99%

Suppressing label noise in medical image classification using mixup attention and self-supervised learning

Gao,

Jiang,

et al. 2024

Phys. Med. Biol.

View full text Add to dashboard Cite

Deep neural networks (DNNs) have been widely applied in medical image classification and achieve remarkable classification performance. These achievements heavily depend on large-scale accurately annotated training data. However, label noise is inevitably introduced in the medical image annotation, as the labeling process heavily relies on the expertise and experience of annotators. Meanwhile, DNNs suffer from overfitting noisy labels, degrading the performance of models. Therefore, in this work, we innovatively devise noise-robust training approach to mitigate the adverse effects of noisy labels in medical image classification. Specifically, we incorporate contrastive learning and intra-group attention mixup strategies into the vanilla supervised learning. The contrastive learning for feature extractor helps to enhance visual representation of DNNs. The intra-group attention mixup module constructs groups and assigns self-attention weights for group-wise samples, and subsequently interpolates massive noisy-suppressed samples through weighted mixup operation. We conduct comparative experiments on both synthetic and real-world noisy medical datasets under various noise levels. Rigorous experiments validate that our noise-robust method with contrastive learning and attention mixup can effectively handle with label noise, and is superior to state-of-the-art methods. An ablation study also shows that both components contribute to boost model performance. The proposed method demonstrates its capability of curb label noise and has certain potential toward real-world clinic applications.

show abstract

Section: Limitations and Future Directionmentioning

confidence: 99%