Cross-Mix Monitoring for Medical Image Segmentation With Limited Supervision

Is denoising necessary for ultrasound image segmentation deep learning: review and benchmark

“…3) Small data alleviation with weak supervision, semisupervision or self-supervision strategies: The current image segmentation algorithms mostly rely on the availability of large amount of training data with pixel-level annotations, which are often expensive, tedious, and laborious. To alleviate the labeling burden and small data limitation, the past years have witnessed an increasing attention in building label-efficient deep segmentation algorithms [12], [70], [71]. According to the supervision provided by different types of labeling, deep learning methods with weak supervision, semi-supervision, or self-supervision strategies are explored to alleviate the problem of medical data limitation in data-driven deep learning.…”

“…Semi-supervised tasks utilize a small amount of well labeled data and a large amount of unlabeled data, learning potential rules from the latter. One approach is based on the prediction consistency on unlabeled data, commonly using the mean-teacher framework [71], [72], [74]. Yang et al [75] added a C-render framework to provide complementary information for the student model.…”

“…Another approach is based on improving pseudo-labels of unlabeled data, for example, Cao et al [77] updated pseudo-labels iteratively with an uncertainty map correction strategy. To gain generalization through adding various approripate perturbations in consistency training, Shu et al [71] proposed cross-network information mixing and transfer mechanism in teaching paradigm to provide extra data flexibility for avoiding lazy student phenomenon and lightweight transductive monitor for connecting the teacherstudent active knowledge distillation.…”

Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentation

“…Huang et al [34] add cutout content loss and slice misalignment as input perturbations. Another common consistency is mix-up consistency [37], [38], [39], which encourages the segmentation of interpolation of two data to be consistent with the interpolation of segmentation results of those data. Apart from disturbances on inputs, there are also many studies focusing on disturbances at feature map level.…”

“…Note that if the perturbations are too weak, it may cause the Lazy Student Phenomenon, but large perturbations may confuse the teacher and student and lead to low performance. Shu et al [38] add a transductive monitor for further knowledge distillation to narrow the semantic gap between the student model and teacher model.…”