Reproducibility of Brain MRI Segmentation Algorithms: Empirical Comparison of Local MAP PSTAPLE, FreeSurfer, and FSL‐FIRST

Abstract

BACKGROUND AND PURPOSE Segmentation of human brain structures is crucial for the volumetric quantification of brain disease. Advances in algorithmic approaches have led to automated techniques that save time compared to interactive methods. Recently, the utility and accuracy of template library fusion algorithms, such as Local MAP PSTAPLE (PSTAPLE), have been demonstrated but there is little guidance regarding its reproducibility compared to single template‐based algorithms such as FreeSurfer and FSL‐FIRST. METHODS Eight repeated magnetic resonance imagings of 20 subjects were segmented using FreeSurfer, FSL‐FIRST, and PSTAPLE. We reported the reproducibility of segmentation‐derived volume measurements for brain structures and calculated sample size estimates for detecting hypothetical rates of tissue atrophy given the observed variances. RESULTS PSTAPLE had the most reproducible volume measurements for hippocampus, putamen, thalamus, caudate, pallidum, amygdala, Accumbens area, and cortical regions. FreeSurfer was most reproducible for brainstem. PSTAPLE was the most accurate algorithm in terms of several metrics include Dice's coefficient. The sample size estimates showed that a study utilizing PSTAPLE would require tens to hundreds less subjects than the other algorithms for detecting atrophy rates typically observed in brain disease. CONCLUSIONS PSTAPLE is a useful tool for automatic human brain segmentation due to its precision and accuracy, which enable the detection of the size of the effect typically reported for neurological disorders with a substantially reduced sample size, in comparison to the other tools we assessed. This enables randomized controlled trials to be executed with reduced cost and duration, in turn, facilitating the assessment of new therapeutic interventions.

Accurate, rapid and reliable, fully automated MRI brainstem segmentation for application in multiple sclerosis and neurodegenerative diseases

“…In particular, the MD‐GRU segmentation approach showed improved reproducibility in the anatomically most challenging region of the BS—the medulla oblongata. Despite relatively high reproducibility compared to other available techniques in earlier studies (Velasco‐Annis et al, ), FreeSurfer segmentations can present inconsistencies especially in the caudal part of the MO. In this part, as well as in the region at or below the level of the central canal entry into the medulla, the differences between medulla and CSF tissue characteristics may lead to anatomically incorrect segmentations.…”

“…A recent study by Velasco‐Annis et al () compared three originally for brain segmentation designed, automated BS segmentation methods (FSL‐FIRST (Patenaude et al, ), PSTAPLE (Akhondi‐Asl & Warfield, ), FreeSurfer (Fischl et al, ; Fischl et al, ; Iglesias et al, ) and reported highest reproducibility of BS segmentations performed by FreeSurfer.…”

“…Manually edited BS segmentations were performed using the open source software 3D Slicer 4.8.1 (http://www.slicer.org), based on presegmentations generated by FreeSurfer (http://surfer.nmr.mgh.harvard.edu//; Iglesias et al, ). We chose FreeSurfer for presegmentations due to its relatively good reproducibility in BS segmentations compared to other methods (Velasco‐Annis, Akhondi‐Asl, Stamm, & Warfield, ).…”

Inter-scanner reproducibility of brain volumetry: influence of automated brain segmentation software

“…segmentation and automatic brain segmentation (ABS) [2]. Both manual and semiautomatic segmentations require manual delineation of brain regions, which are unavoidably susceptible to intra-and inter-rater inconsistency [2,3]. In contrast, ABS is hand-free and thus more resistant to inter-rater variability.…”

“…The atlas that contains brain anatomy information is used as a reference for the segmentation of new MRI images [3]. Labels of brain regions from the atlas are modulated by affine transformations to fit target images [2]. FreeSurfer encompasses template registration and segmentation, and it can measure not only the volumes of many anatomical structures [13] but also other brain structural features such as cortical thickness, surface area, intensity and curvature.…”