Revisiting the Evaluation of Segmentation Results: Introducing Confidence Maps

Restif, Christophe

doi:10.1007/978-3-540-75759-7_71

Cited by 6 publications

(6 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Popular methods for segmentation evaluation [2], [3] compute global scores over the entire image. However, it has been suggested [4] that evaluating local performance of a segmentation algorithm is better suited in some cases, as in some applications the requirements for accuracy vary across the image: very precise delineations may be needed in crucial areas while a lower precision may be acceptable for other areas. New techniques for local performance estimation are critical for such applications, in order to facilitate the automatic and quantitative assessment of segmentation accuracy while incorporating information from multiple experts.…”

Section: Introductionmentioning

confidence: 99%

Estimating A Reference Standard Segmentation With Spatially Varying Performance Parameters: Local MAP STAPLE

Commowick

Akhondi-Asl

Warfield

2012

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

We present a new algorithm, called local MAP STAPLE, to estimate from a set of multi-label segmentations both a reference standard segmentation and spatially varying performance parameters. It is based on a sliding window technique to estimate the segmentation and the segmentation performance parameters for each input segmentation. In order to allow for optimal fusion from the small amount of data in each local region, and to account for the possibility of labels not being observed in a local region of some (or all) input segmentations, we introduce prior probabilities for the local performance parameters through a new Maximum A Posteriori formulation of STAPLE. Further, we propose an expression to compute confidence intervals in the estimated local performance parameters. We carried out several experiments with local MAP STAPLE to characterize its performance and value for local segmentation evaluation. First, with simulated segmentations with known reference standard segmentation and spatially varying performance, we show that local MAP STAPLE performs better than both STAPLE and majority voting. Then we present evaluations with data sets from clinical applications. These experiments demonstrate that spatial adaptivity in segmentation performance is an important property to capture. We compared the local MAP STAPLE segmentations to STAPLE, and to previously published fusion techniques and demonstrate the superiority of local MAP STAPLE over other state-of-the- art algorithms.

show abstract

Section: Introductionmentioning

confidence: 99%

Estimating A Reference Standard Segmentation With Spatially Varying Performance Parameters: Local MAP STAPLE

Commowick

Akhondi-Asl

Warfield

2012

IEEE Trans. Med. Imaging

View full text Add to dashboard Cite

show abstract

“…The necessity of local measure use was signaled by Restif (2007). He proposed the confidence maps estimating true segmentation (COMETS) framework.…”

Section: Related Workmentioning

confidence: 99%

LEFMIS: locally-oriented evaluation framework for medical image segmentation algorithms

Skalski

Jakubowski²,

Drewniak

2018

Phys. Med. Biol.

View full text Add to dashboard Cite

This article proposes a novel framework for the locally-oriented evaluation of segmentation algorithms (LEFMIS). The presented approach is robust and takes into account local inter/intra-observer variability and the anisotropy of medical images. What is more, the framework makes it possible to distinguish types of error locally. These features are crucial in the context of cancer image data. The proposed framework is based on use of the signed anisotropic Euclidean distance transform and the distance projection. It can be used easily in many different applications with or without additional expert outlines (both inter- and intra-observer variability). The performance of the proposed framework is depicted using both artificial and kidney cancer CT data with experts' manual outlines. In the article, in the case of artificial data, it is presented that the manual outlines dispersion is symmetric in relation to the truth border. The effectiveness of the selected segmentation algorithm was analysed in the context of kidney cancer using computed tomography data. For the calculated local inter-observer variability, 80.11% of the surface points generated by the kidney segmentation algorithm are within one expert outline standard deviation and 97.96% are within five. An error distribution shift in the direction of type I error equivalent was also observed. Finally, the significance of the local estimation of error type differences is presented. The article shows the greater usefulness and flexibility of the proposed framework in comparison to the state-of-the-art methods. The exemplary usage of the LEFMIS with or without inter-/intra-observer variability is also presented.

show abstract

“…Global scores of segmentation quality for label fusion were proposed in [2,3]. However, as suggested by Restif in [4] the computation of local performance is a better measure since it suits applications requiring varying accuracy in different image areas. Majority voting has also been used for fusing atlases of the brain in [5].…”

Section: Related Workmentioning

confidence: 99%

Consensus Based Medical Image Segmentation Using Semi-Supervised Learning And Graph Cuts

Mahapatra¹

2016

Preprint

View full text Add to dashboard Cite

Medical image segmentation requires consensus ground truth segmentations to be derived from multiple expert annotations. A novel approach is proposed that obtains consensus segmentations from experts using graph cuts (GC) and semi supervised learning (SSL). Popular approaches use iterative Expectation Maximization (EM) to estimate the final annotation and quantify annotator's performance. Such techniques pose the risk of getting trapped in local minima. We propose a self consistency (SC) score to quantify annotator consistency using low level image features. SSL is used to predict missing annotations by considering global features and local image consistency. The SC score also serves as the penalty cost in a second order Markov random field (MRF) cost function optimized using graph cuts to derive the final consensus label. Graph cut obtains a global maximum without an iterative procedure. Experimental results on synthetic images, real data of Crohn's disease patients and retinal images show our final segmentation to be accurate and more consistent than competing methods.

show abstract

Revisiting the Evaluation of Segmentation Results: Introducing Confidence Maps

Cited by 6 publications

References 15 publications

Estimating A Reference Standard Segmentation With Spatially Varying Performance Parameters: Local MAP STAPLE

Estimating A Reference Standard Segmentation With Spatially Varying Performance Parameters: Local MAP STAPLE

LEFMIS: locally-oriented evaluation framework for medical image segmentation algorithms

Consensus Based Medical Image Segmentation Using Semi-Supervised Learning And Graph Cuts

Contact Info

Product

Resources

About