A toolbox for multiple sclerosis lesion segmentation

We present a study of multiple sclerosis segmentation algorithms conducted at the international MICCAI 2016 challenge. This challenge was operated using a new open-science computing infrastructure. This allowed for the automatic and independent evaluation of a large range of algorithms in a fair and completely automatic manner. This computing infrastructure was used to evaluate thirteen methods of MS lesions segmentation, exploring a broad range of state-of-theart algorithms, against a high-quality database of 53 MS cases coming from four centers following a common definition of the acquisition protocol. Each case was annotated manually by an unprecedented number of seven different experts. Results of the challenge highlighted that automatic algorithms, including the recent machine learning methods (random forests, deep learning, …), are still trailing human expertise on both detection and delineation criteria. In addition, we demonstrate that computing a statistically robust consensus of the algorithms performs closer to human expertise on one score (segmentation) although still trailing on detection scores.

show abstract

“…RouraX. LladóOutlier segmentation based on brain tissue labeling and post-processing rules 35,36 CPU T 1 -w, FLAIR (raw)9M. SantosA.…”

Section: Resultsmentioning

confidence: 99%

Objective Evaluation of Multiple Sclerosis Lesion Segmentation using a Data Management and Processing Infrastructure

Commowick

Istace

Kain

et al. 2018

Sci Rep

Self Cite

217

184

View full text Add to dashboard Cite

show abstract

“…Roy et al (2015a) extended their 3D example patch-based lesion segmentation algorithm to 4D by considering a time series of patches from available training data. Other work evaluated WML changes over time (Battaglini et al, 2014; Elliott et al, 2010; Ganiler et al, 2014; Roura et al, 2015; Sweeney et al, 2013a) with the focus being on the appearance/disappearance of lesions by subtraction of the intensity images of consecutive time-points. As there clearly has been a relative dearth of work on the automated segmentation of time-series WMLs, and as there is no approach that has gained widespread acceptance, a main purpose of this paper is to provide a public database to reignite work in this area.…”

Section: Introductionmentioning

confidence: 99%

Longitudinal multiple sclerosis lesion segmentation: Resource and challenge

et al. 2017

View full text Add to dashboard Cite

In conjunction with the ISBI 2015 conference, we organized a longitudinal lesion segmentation challenge providing training and test data to registered participants. The training data consisted of five subjects with a mean of 4.4 time-points, and test data of fourteen subjects with a mean of 4.4 time-points. All 82 data sets had the white matter lesions associated with multiple sclerosis delineated by two human expert raters. Eleven teams submitted results using state-of-the-art lesion segmentation algorithms to the challenge, with ten teams presenting their results at the conference. We present a quantitative evaluation comparing the consistency of the two raters as well as exploring the performance of the eleven submitted results in addition to three other lesion segmentation algorithms. The challenge presented three unique opportunities: 1) the sharing of a rich data set; 2) collaboration and comparison of the various avenues of research being pursued in the community; and 3) a review and refinement of the evaluation metrics currently in use. We report on the performance of the challenge participants, as well as the construction and evaluation of a consensus delineation. The image data and manual delineations will continue to be available for download, through an evaluation website 1 as a resource for future researchers in the area. This data resource provides a platform to compare existing methods in a fair and consistent manner to each other and multiple manual raters.

show abstract

“…This is confirmed by another recent study on lesion segmentation in a multi-center data set, reporting similar (low) performances (range of SI=0.11-0.45) compared to the current study. 28 The use of a multi-center data set is a strength of this study, providing a better approximation of the clinical setting, and moreover allowing us to show that the automated segmentation methods are robust to new data of an 'unseen' center. Summarizing, the four methods are robust, however, not accurate enough on spatial agreement.…”

Section: Discussionmentioning

confidence: 99%

Performance of five research-domain automated WM lesion segmentation methods in a multi-center MS study

Sitter

Steenwijk²,

Ruet

et al. 2017

NeuroImage

View full text Add to dashboard Cite

Background and Purpose: In vivo identification of white matter lesions plays a key-role in evaluation of patients with multiple sclerosis (MS). Automated lesion segmentation methods have been developed to substitute manual outlining, but evidence of their performance in multi-center investigations is lacking. In this work, five research-domain automated segmentation methods were evaluated using a multi-center MS dataset. Positive (FP) volumes and False Negative (FN) volumes were used to examine spatial agreement. All analyses were repeated using a leave-one-center-out design to exclude the center of interest from the training phase to evaluate the performance of the method on 'unseen' center.

show abstract

A toolbox for multiple sclerosis lesion segmentation

Abstract: Our tool is implemented as a publicly available SPM8/12 extension that can be used by both the medical and research communities.

Cited by 88 publications

References 29 publications

Objective Evaluation of Multiple Sclerosis Lesion Segmentation using a Data Management and Processing Infrastructure

Objective Evaluation of Multiple Sclerosis Lesion Segmentation using a Data Management and Processing Infrastructure

Longitudinal multiple sclerosis lesion segmentation: Resource and challenge

Performance of five research-domain automated WM lesion segmentation methods in a multi-center MS study

Contact Info

Product

Resources

About