Segmentation editing improves efficiency while reducing inter-expert variation and maintaining accuracy for normal brain tissues in the presence of space-occupying lesions

Abstract

Image segmentation has become a vital and often rate limiting step in modern radiotherapy treatment planning. In recent years the pace and scope of algorithm development, and even introduction into the clinic, have far exceeded evaluative studies. In this work we build upon our previous evaluation of a registration driven segmentation algorithm in the context of 8 expert raters and 20 patients who underwent radiotherapy for large space-occupying tumors in the brain. In this work we tested four hypotheses concerning the impact of manual segmentation editing in a randomized single-blinded study. We tested these hypotheses on the normal structures of the brainstem, optic chiasm, eyes and optic nerves using the Dice similarity coefficient, volume, and signed Euclidean distance error to evaluate the impact of editing on inter-rater variance and accuracy. Accuracy analyses relied on two simulated ground truth estimation methods: STAPLE and a novel implementation of probability maps. The experts were presented with automatic, their own, and their peers’ segmentations from our previous study to edit. We found, independent of source, editing reduced inter-rater variance while maintaining or improving accuracy and improving efficiency with at least 60% reduction in contouring time. In areas where raters performed poorly contouring from scratch, editing of the automatic segmentations reduced the prevalence of total anatomical miss from approximately 16% to 8% of the total slices contained within the ground truth estimations. These findings suggest that contour editing could be useful for consensus building such as in developing delineation standards, and that both automated methods and even perhaps less sophisticated atlases could improve efficiency, inter-rater variance, and accuracy.

Automatic Intracranial Segmentation: Is the Clinician Still Needed?

“… 8 Using atlas-based registration to segment the eyes and the brain stem and the atlas-navigated optimal medial axis and deformable model algorithm for other structures followed by manual editing, they then showed in another study that it could reduce interobserver variation, improve accuracy, and reduce segmentation time by about 60%. 9 …”

“…If automated segmentation alone was accurate, this could reduce segmentation time. However, the physician is still needed to further segment structures, 4 , 9 especially in the context of inverse-planned hypofractionated radiation therapy. Using both an automated software and manual editing may be the current best of both worlds to deliver safe treatment to our patients.…”

Multi-Modal Glioblastoma Segmentation: Man versus Machine

“…Several studies reported failures of individual expert rater segmentations because structural MRI may obscure the precise visual delineation of glioma boundaries [10] – [12] . In addition, manual segmentation is time-consuming and is a rate-determining step for further treatment planning [13] . Fully automated user-independent segmentation tools are available for research purposes.…”

Multi-modal Glioblastoma Segmentation: Man versus Machine

“…Several studies reported failures of individual expert rater segmentations because structural MRI may obscure the precise visual delineation of glioma boundaries [10][11][12]. In addition, manual segmentation is time-consuming and is a rate-determining step for further treatment planning [13]. Fully automated user-independent segmentation tools are available for research purposes.…”