Medical Matting: A New Perspective on Medical Segmentation with Uncertainty

Abstract

In medical image segmentation, it is difficult to mark ambiguous areas accurately with binary masks, especially when dealing with small lesions. Therefore, it is a challenge for radiologists to reach a consensus by using binary masks under the condition of multiple annotations. However, these areas may contain anatomical structures that are conducive to diagnosis. Uncertainty is introduced to study these situations. Nevertheless, the uncertainty is usually measured by the variances between predictions in a multiple trial way. It is not intuitive, and there is no exact correspondence in the image. Inspired by image matting, we introduce matting as a soft segmentation method and a new perspective to deal with and represent uncertain regions into medical scenes, namely medical matting. More specifically, because there is no available medical matting dataset, we first labeled two medical datasets with alpha matte. Secondly, the matting method applied to the natural image is not suitable for the medical scene, so we propose a new architecture to generate binary masks and alpha matte in a row. Thirdly, the uncertainty map is introduced to highlight the ambiguous regions from the binary results and improve the matting performance. Evaluated on these datasets, the proposed model outperformed state-of-the-art matting algorithms by a large margin, and alpha matte is proved to be a more efficient labeling form than a binary mask.

Uncertainty-aware Multi-modal Learning via Cross-modal Random Network Prediction

“…Jungo et al [17] used two medical datasets to compare several uncertainty measurement models, namely: softmax entropy [12], Monte Carlo dropout [12], aleatoric uncertainty [18], ensemble methods [21] and auxiliary network [8,31]. In MIA, multiple uncertainty measurements have been proposed as well [22,24,36,37]. However, none of the methods above are designed for multi-modal tasks and some of them contain long and complex pipelines that are not easily adaptable to new tasks.…”

3D Matting: A Soft Segmentation Method Applied in Computed Tomography

“…Image matting [17,18,19,20,21,14,22,23,24] uses the mixing coefficient α, also known as the alpha matte, to decompose the image I to foreground F and background B, or lesions and its surrounding tissues in medical images [2], which can be defined as…”

“…Due to image noises, the occlusion of human tissues, the principles of medical imaging, and the anatomical structure characteristics of lesions, fuzzy boundaries are inevitable and ubiquitous in medical images [1,2]. In segmentation tasks, binary masks are the most commonly used to identify lesion regions.…”

“…Instead of identifying perfect lesion boundaries that are difficult to annotate using binarystyle masks, medical matting [2] makes use of the ambiguous regions. It introduces image matting into the medical scenes, and regards medical images as a mixture of lesions and healthy tissues.…”

“…The previous works mainly focused on 2D medical images [2,8,9], and the existing research on 3D medical image matting is very limited in quantity and methodology. To the best of our knowledge, only [10,11,12,13] have touched upon the problem of the 3D matting, and they are mainly based on 2D closed-form matting [14].…”