Imperfect Segmentation Labels: How Much Do They Matter?

Heller, Nicholas; Deán, J. Luis; Papanikolopoulos, Nikolaos

doi:10.1007/978-3-030-01364-6_13

Cited by 25 publications

(13 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This strategy is useful in mitigating the disadvantages of a trade-off between mask quality and big sample size, as encountered when using automatically generated training data ( McClure et al, 2019 ). This is further reinforced by the robust performance of the trained CNN here, demonstrating that the approximate masks despite individual flaws allow for a successful capture of the structure’s properties by the neural networks ( Heller et al, 2018 ).…”

Section: Discussionmentioning

confidence: 54%

“…Although the 3D version involves a higher computational load which may limit the upper resolution of processed images, the inclusion of additional dimension was shown to be of significant benefit to the segmentation ( Chen et al, 2019 ; Mlynarski et al, 2020 ). Another reason for using the U-Net architecture was its reported robustness to jagged boundary-localized errors ( Heller et al, 2018 ), which is a helpful feature in case of training on automatically generated masks.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning-Based Detection of Malformed Optic Chiasms From MRI Images

2021

View full text Add to dashboard Cite

Convolutional neural network (CNN) models are of great promise to aid the segmentation and analysis of brain structures. Here, we tested whether CNN trained to segment normal optic chiasms from the T1w magnetic resonance imaging (MRI) image can be also applied to abnormal chiasms, specifically with optic nerve misrouting as typical for human albinism. We performed supervised training of the CNN on the T1w images of control participants (n = 1049) from the Human Connectome Project (HCP) repository and automatically generated algorithm-based optic chiasm masks. The trained CNN was subsequently tested on data of persons with albinism (PWA; n = 9) and controls (n = 8) from the CHIASM repository. The quality of outcome segmentation was assessed via the comparison to manually defined optic chiasm masks using the Dice similarity coefficient (DSC). The results revealed contrasting quality of masks obtained for control (mean DSC ± SEM = 0.75 ± 0.03) and PWA data (0.43 ± 0.8, few-corrected p = 0.04). The fact that the CNN recognition of the optic chiasm fails for chiasm abnormalities in PWA underlines the fundamental differences in their spatial features. This finding provides proof of concept for a novel deep-learning-based diagnostics approach of chiasmal misrouting from T1w images, as well as further analyses on chiasmal misrouting and their impact on the structure and function of the visual system.

show abstract

Section: Discussionmentioning

confidence: 54%

Section: Methodsmentioning

confidence: 99%

Deep Learning-Based Detection of Malformed Optic Chiasms From MRI Images

2021

View full text Add to dashboard Cite

show abstract

“…Moreover, we acknowledge that manually correcting brain masks in a single case can take hours (Puccio et al, ). Although our approach with generating a GT brain mask in a large‐scale dataset was more focused on correcting major errors (e.g., around pathologies, resection cavities or due to varying hardware or acquisition parameters), even imperfect GT labels can lead to high quality deep‐learning segmentation algorithms when using the UNET architecture that was employed in our study (Heller, Dean, & Papanikolopoulos, ). Moreover, the competitiveness of our approach was rendered by testing on the public datasets (NFBS, CC‐359, and LPBA40) where we confirmed the performance of the HD‐BET algorithm against an independent high‐quality GT.…”

Section: Discussionmentioning

confidence: 99%

Automated brain extraction of multisequence MRI using artificial neural networks

Isensee

Schell

Pflueger

et al. 2019

Human Brain Mapping

432

253

View full text Add to dashboard Cite

Brain extraction is a critical preprocessing step in the analysis of neuroimaging studies conducted with magnetic resonance imaging (MRI) and influences the accuracy of downstream analyses. The majority of brain extraction algorithms are, however, optimized for processing healthy brains and thus frequently fail in the presence of pathologically altered brain or when applied to heterogeneous MRI datasets. Here we introduce a new, rigorously validated algorithm (termed HD‐BET) relying on artificial neural networks that aim to overcome these limitations. We demonstrate that HD‐BET outperforms six popular, publicly available brain extraction algorithms in several large‐scale neuroimaging datasets, including one from a prospective multicentric trial in neuro‐oncology, yielding state‐of‐the‐art performance with median improvements of +1.16 to +2.50 points for the Dice coefficient and −0.66 to −2.51 mm for the Hausdorff distance. Importantly, the HD‐BET algorithm, which shows robust performance in the presence of pathology or treatment‐induced tissue alterations, is applicable to a broad range of MRI sequence types and is not influenced by variations in MRI hardware and acquisition parameters encountered in both research and clinical practice. For broader accessibility, the HD‐BET prediction algorithm is made freely available (http://www.neuroAI-HD.org) and may become an essential component for robust, automated, high‐throughput processing of MRI neuroimaging data.

show abstract

“…Although deep-learning is relatively robust to label noise, 48 perturbations along boundaries are particularly problematic for U-Net model training. 49 In prior studies, correcting contours, as opposed to contouring from scratch has been shown to increase consistency and reduce inter-observer variation. 50 The two outliers with low dice scores were in patients with anatomical abnormalities that will also likely be correlated to high dose to the heart (Figure 7) and in one case would have resulted in an error in dose quantification of over 6 Gray had it not been corrected (Figure 10).…”

Section: Discussionmentioning

confidence: 99%

RootPainter3D: Interactive‐machine‐learning enables rapid and accurate contouring for radiotherapy

et al. 2021

View full text Add to dashboard Cite

Purpose: Organ-at-risk contouring is still a bottleneck in radiotherapy,with many deep learning methods falling short of promised results when evaluated on clinical data. We investigate the accuracy and time-savings resulting from the use of an interactive-machine-learning method for an organ-at-risk contouring task. Methods:We implement an open-source interactive-machine-learning software application that facilitates corrective-annotation for deep-learning generated contours on X-ray CT images. A trained-physician contoured 933 hearts using our software by delineating the first image, starting model training, and then correcting the model predictions for all subsequent images. These corrections were added into the training data, which was used for continuously training the assisting model. From the 933 hearts, the same physician also contoured the first 10 and last 10 in Eclipse (Varian) to enable comparison in terms of accuracy and duration. Results: We find strong agreement with manual delineations, with a dice score of 0.95. The annotations created using corrective-annotation also take less time to create as more images are annotated, resulting in substantial time savings compared to manual methods. After 923 images had been delineated, hearts took 2 min and 2 s to delineate on average, which includes time to evaluate the initial model prediction and assign the needed corrections, compared to 7 min and 1 s when delineating manually. Conclusions: Our experiment demonstrates that interactive-machine-learning with corrective-annotation provides a fast and accessible way for non computerscientists to train deep-learning models to segment their own structures of interest as part of routine clinical workflows.

show abstract

Imperfect Segmentation Labels: How Much Do They Matter?

Cited by 25 publications

References 20 publications

Deep Learning-Based Detection of Malformed Optic Chiasms From MRI Images

Deep Learning-Based Detection of Malformed Optic Chiasms From MRI Images

Automated brain extraction of multisequence MRI using artificial neural networks

RootPainter3D: Interactive‐machine‐learning enables rapid and accurate contouring for radiotherapy

Contact Info

Product

Resources

About