SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining

Billot, Benjamin; Greve, Douglas N.; Puonti, Oula; Thielscher, Axel; Leemput, Koen Van; Fischl, Bruce; Dalca, Adrian V.; Iglesias, Juan Eugenio

doi:10.48550/arxiv.2107.09559

Cited by 11 publications

(23 citation statements)

References 94 publications

(144 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We evaluate SynthSeg + (i.e., S 1 + D + S 2 ) against four approaches. SynthSeg [6]: We use the publicly available model for testing. Cascaded networks [35] (S 1 + S 2 ): we ablate the denoiser D to obtain an architecture that is representative of classical cascaded networks.…”

Section: Competing Methodsmentioning

confidence: 99%

“…As a result, S 1 and S 2 are exposed to vastly varying examples, which forces them to learn contrast-and resolution-agnostic features. Training imagetarget pairs for S 1 are generated with a procedure similar to SynthSeg [6]:…”

Section: Training Scheme For the Segmentation Modulesmentioning

confidence: 99%

“…All modules are trained separately using the average soft Dice as loss function [28]. The segmentation modules S 1 and S 2 use the same 3D UNet architec-ture as in [6]. Briefly, it comprises 5 levels of 2 convolution layers.…”

Section: Implementation Detailsmentioning

confidence: 99%

“…Very recently, a publicly available method named SynthSeg [6] has been proposed for out-of-the-box segmentation of brain scans of any contrast and resolution. SynthSeg relies on a 3D UNet trained on synthetic scans generated with a domain randomisation approach [36].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Robust Segmentation of Brain MRI in the Wild with Hierarchical CNNs and no Retraining

Billot¹,

Colin²,

Arnold³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Retrospective analysis of brain MRI scans acquired in the clinic has the potential to enable neuroimaging studies with sample sizes much larger than those found in research datasets. However, analysing such clinical images "in the wild" is challenging, since subjects are scanned with highly variable protocols (MR contrast, resolution, orientation, etc.). Nevertheless, recent advances in convolutional neural networks (CNNs) and domain randomisation for image segmentation, best represented by the publicly available method SynthSeg, may enable morphometry of clinical MRI at scale. In this work, we first evaluate SynthSeg on an uncurated, heterogeneous dataset of more than 10,000 scans acquired at Massachusetts General Hospital. We show that SynthSeg is generally robust, but frequently falters on scans with low signal-to-noise ratio or poor tissue contrast. Next, we propose SynthSeg + , a novel method that greatly mitigates these problems using a hierarchy of conditional segmentation and denoising CNNs. We show that this method is considerably more robust than SynthSeg, while also outperforming cascaded networks and state-of-the-art segmentation denoising methods. Finally, we apply our approach to a proof-of-concept volumetric study of ageing, where it closely replicates atrophy patterns observed in research studies conducted on high-quality, 1 mm, T1-weighted scans. The code and trained model are publicly available at https://github.com/BBillot/SynthSeg.

show abstract

Section: Competing Methodsmentioning

confidence: 99%

Section: Training Scheme For the Segmentation Modulesmentioning

confidence: 99%

Section: Implementation Detailsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Robust Segmentation of Brain MRI in the Wild with Hierarchical CNNs and no Retraining

Billot¹,

Colin²,

Arnold³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…To further understand the potential of MR images in predicting Alzheimer's prediction, several single modality single task modeling experiments are conducted by using volumetric features including brain volume, hippocampus volume, and ventricle volume that are extracted either by FreeSurfer v6.0 [7,20] or SynthSeg [2]. The results in Figure 3 show that extracted volumetric features do not provide compelling accuracy in predicting cognitive endpoints neither in single task nor in multitask setups, indicating that they do not take advantage of correlation among cognitive scores.…”

Section: Raw Mri Vs Hard-coded Featuresmentioning

confidence: 99%

DeepAD: A Robust Deep Learning Model of Alzheimer's Disease Progression for Real-World Clinical Applications

Hashemifar¹,

Iriondo²,

Casey³

et al. 2022

Preprint

View full text Add to dashboard Cite

The ability to predict the future trajectory of a patient is a key step toward the development of therapeutics for complex diseases such as Alzheimer's disease (AD). However, most machine learning approaches developed for prediction of disease progression are either single-task or singlemodality models, which can not be directly adopted to our setting involving multi-task learning with high dimensional images. Moreover, most of those approaches are trained on a single dataset (i.e. cohort), which can not be generalized to other cohorts. We propose a novel multimodal multi-task deep learning model to predict AD progression by analyzing longitudinal clinical and neuroimaging data from multiple cohorts. Our proposed model integrates high dimensional MRI features from a 3D convolutional neural network with other data modalities, including clinical and demographic information, to predict the future trajectory of patients. Our model employs an adversarial loss to alleviate the studyspecific imaging bias, in particular the inter-study domain shifts. In addition, a Sharpness-Aware Minimization (SAM) optimization technique is applied to further improve model generalization. The proposed model is trained and tested on various datasets in order to evaluate and validate the results. Our results showed that 1) our model yields significant improvement over the baseline models, and 2) models using extracted neuroimaging features from 3D convolutional neural network outperform the same models when applied to MRI-derived volumetric features.

show abstract

Fitting Segmentation Networks on Varying Image Resolutions Using Splatting

Brudfors

Balbastre²,

Ashburner³

et al. 2022

Medical Image Understanding and Analysis

View full text Add to dashboard Cite

Data used in image segmentation are not always defined on the same grid. This is particularly true for medical images, where the resolution, field-of-view and orientation can differ across channels and subjects. Images and labels are therefore commonly resampled onto the same grid, as a pre-processing step. However, the resampling operation introduces partial volume effects and blurring, thereby changing the effective resolution and reducing the contrast between structures. In this paper we propose a splat layer, which automatically handles resolution mismatches in the input data. This layer pushes each image onto a mean space where the forward pass is performed. As the splat operator is the adjoint to the resampling operator, the mean-space prediction can be pulled back to the native label space, where the loss function is computed. Thus, the need for explicit resolution adjustment using interpolation is removed. We show on two publicly available datasets, with simulated and real multi-modal magnetic resonance images, that this model improves segmentation results compared to resampling as a pre-processing step.

show abstract

SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining

Cited by 11 publications

References 94 publications

Robust Segmentation of Brain MRI in the Wild with Hierarchical CNNs and no Retraining

Robust Segmentation of Brain MRI in the Wild with Hierarchical CNNs and no Retraining

DeepAD: A Robust Deep Learning Model of Alzheimer's Disease Progression for Real-World Clinical Applications

Fitting Segmentation Networks on Varying Image Resolutions Using Splatting

Contact Info

Product

Resources

About