Supervised Uncertainty Quantification for Segmentation with Multiple Annotations

Shi, Hongyin; Worrall, Daniel E.; Knegt, Stefan J. L.; Veeling, Bastiaan S.; Huisman, Henkjan; Welling, Max

doi:10.1007/978-3-030-32245-8_16

Cited by 81 publications

(66 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, the evaluation was carried out on a single dataset only. Several extensions and modifications of the Probabilistic U-Net have been published recently: Hu et al 24 introduced variational dropout 25 after the last convolutional layer of the U-Net to estimate epistemic uncertainty in the produced segmentations. In 26 , the original authors of the Probabilistic U-Net improved their work by proposing a hierarchical latent space decomposition, which aimed at improving segmentation diversity by modelling the segmentation distribution at various scales.…”

mentioning

confidence: 99%

Radiomics feature reproducibility under inter-rater variability in segmentations of CT images

Haarburger

Müller‐Franzes

Weninger

et al. 2020

Sci Rep

View full text Add to dashboard Cite

identifying image features that are robust with respect to segmentation variability is a tough challenge in radiomics. So far, this problem has mainly been tackled in test-retest analyses. in this work we analyse radiomics feature reproducibility in two phases: first with manual segmentations provided by four expert readers and second with probabilistic automated segmentations using a recently developed neural network (pHiseg). We test feature reproducibility on three publicly available datasets of lung, kidney and liver lesions. We find consistent results both over manual and automated segmentations in all three datasets and show that there are subsets of radiomic features which are robust against segmentation variability and other radiomic features which are prone to poor reproducibility under differing segmentations. By providing a detailed analysis of robustness of the most common radiomics features across several datasets, we envision that more reliable and reproducible radiomic models can be built in the future based on this work.

show abstract

mentioning

confidence: 99%

Radiomics feature reproducibility under inter-rater variability in segmentations of CT images

Haarburger

Müller‐Franzes

Weninger

et al. 2020

Sci Rep

View full text Add to dashboard Cite

show abstract

“…In this study, our sophisticated data acquisition procedure has reduced the aleatoric uncertainty to a relatively low extent, as verified by visual assessments. Multiple annotations in a probabilistic setting may help calibrate the estimation of aleatoric uncertainty 32 , which will be one of our future endeavors.…”

Section: Data Recordsmentioning

confidence: 99%

The SUSTech-SYSU dataset for automatically segmenting and classifying corneal ulcers

et al. 2020

View full text Add to dashboard Cite

Corneal ulcer is a common ophthalmic symptom. Segmentation algorithms are needed to identify and quantify corneal ulcers from ocular staining images. Developments of such algorithms have been obstructed by a lack of high quality datasets (the ocular staining images and the corresponding goldstandard ulcer segmentation labels), especially for supervised learning based segmentation algorithms. In such context, we prepare a dataset containing 712 ocular staining images and the associated segmentation labels of flaky corneal ulcers. In addition to segmentation labels for flaky corneal ulcers, we also provide each image with three-fold class labels: firstly, each image has a label in terms of its general ulcer pattern; secondly, each image has a label in terms of its specific ulcer pattern; thirdly, each image has a label indicating its ulcer severity degree. This dataset not only provides an excellent opportunity for investigating the accuracy and reliability of different segmentation and classification algorithms for corneal ulcers, but also advances the development of new supervised learning based algorithms especially those in the deep learning framework.

show abstract

“…A common practice in the literature is to collect multiple annotations per sample and produce determined training samples with label fusion such as majority voting. This approach is particularly useful when inter-rater agreement is expected to be low [59]. A recent work [59] exploits IOV, where the uncertainty brought by IOV is treated as a target in supervised learning problem [58].…”

Section: Discussionmentioning

confidence: 99%

“…This approach is particularly useful when inter-rater agreement is expected to be low [59]. A recent work [59] exploits IOV, where the uncertainty brought by IOV is treated as a target in supervised learning problem [58]. We conclude that since the evaluation of curvilinear structure segmentation is pixel-sensitive, it can be expected that the standardization of annotated positions on targeted curvilinear structures can be very helpful to increase the pixellevel agreements of multiple observers, and hence reduce the aleatoric uncertainty in IOV.…”

Section: Discussionmentioning

confidence: 99%

Automated Tortuosity Analysis of Nerve Fibers in Corneal Confocal Microscopy

Zhao

Zhang

Pereira

et al. 2020

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

Precise characterization and analysis of corneal nerve fiber tortuosity are of great importance in facilitating examination and diagnosis of many eye-related diseases. In this paper we propose a fully automated method for image-level tortuosity estimation, comprising image enhancement, exponential curvature estimation, and tortuosity level classification. The image enhancement component is based on an extended Retinex model, which not only corrects imbalanced illumination and improves image contrast in an image, but also models noise explicitly to aid removal of imaging noise. Afterwards, we take advantage of exponential curvature estimation in the 3D space of positions and orientations to directly measure curvature based on the enhanced images, rather than relying on the explicit segmentation and skeletonization steps in a conventional pipeline usually with accumulated preprocessing errors. The proposed method has been applied over two corneal nerve microscopy datasets for the estimation of a tortuosity level for each image. The experimental results show that it performs better than several selected state-of-the-art methods. Furthermore, we have performed manual gradings at tortuosity level of four hundred and three corneal nerve microscopic images, and this dataset has been released for public access to facilitate other researchers in the community in carrying out further research on the same and related topics.

show abstract

Supervised Uncertainty Quantification for Segmentation with Multiple Annotations

Cited by 81 publications

References 14 publications

Radiomics feature reproducibility under inter-rater variability in segmentations of CT images

Radiomics feature reproducibility under inter-rater variability in segmentations of CT images

The SUSTech-SYSU dataset for automatically segmenting and classifying corneal ulcers

Automated Tortuosity Analysis of Nerve Fibers in Corneal Confocal Microscopy

Contact Info

Product

Resources

About