The WORC database: MRI and CT scans, segmentations, and clinical labels for 930 patients from six radiomics studies

Starmans, Martijn P.A.; Timbergen, Milea J M; Vos, Melissa; Padmos, Guillaume A.; Grünhagen, Dirk J.; Verhoef, Cornelis; Sleijfer, Stefan; Leenders, Geert J.L.H. van; Buisman, Florian E.; Willemssen, F.; Koerkamp, B. Groot; Angus, Lindsay; Veldt, Astrid A.M. van der; Rajicic, Ana; Odink, Arlette E.; Renckens, Michel; Doukas, Michail; Man, Rob A. de; IJzermans, Jan N. M.; Miclea, Razvan L.; Vermeulen, Peter B.; Thomeer, Maarten; Visser, Jacob J.; Niessen, Wiro J.; Klein, Stefan

doi:10.1101/2021.08.19.21262238

Cited by 15 publications

(17 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A recent study (dos Santos et al, 2021) even warned that radiomics research must achieve "higher evidence levels" to avoid a reproducibility crisis such as the recent one in psychology (Open Science Collaboration, 2015). Hence, to facilitate reproducibility, besides automating the radiomics workflows construction, we have publicly released six datasets with a total of 930 patients (Starmans et al, 2021c), and made the WORC toolbox and the code to perform our experiments on all datasets open-source (Starmans et al, 2018a;Starmans, 2021). Besides a lack of reproducibility, there is a positive publication bias in radiomics, with as few as 6% of the studies between 2015 and 2018 showing negative results as reported by Buvat and Orlhac (2019).…”

Section: Discussionmentioning

confidence: 99%

“…The first six datasets (Lipo, Desmoid, Liver, GIST, CRLM, and Melanoma) are publicly released as part of this study, see (Starmans et al, 2021c) for more details. Three datasets (HCC, MesFib, and Prostate) cannot be made publicly available.…”

Section: Evaluation Of Default Configuration On Twelve Different Clin...mentioning

confidence: 99%

“…For the Glioma dataset, the raw imaging data was not avail- O Dataset publicly released as part of this study (Starmans et al, 2021c). ); e. colorectal liver metastases (Starmans et al, 2021a); f. melanoma (Angus et al, 2021); g. hepatocellular carcinoma (Starmans et al, 2020a); h. mesenteric fibrosis (Blazevic et al, 2021); i. prostate cancer (Castillo T et al, 2019); j. low grade glioma (van der Voort et al, 2019b); k. Alzheimer's disease (Bron et al, 2021); and l. head and neck cancer (Aerts et al, 2014).…”

Section: Evaluation Of Default Configuration On Twelve Different Clin...mentioning

confidence: 99%

“…Six of the datasets used in this study (Lipo, Desmoid, Liver, GIST, CRLM, and Melanoma), comprising a total of 930 patients, are publicly released as part of this study and hosted via a public XNAT 3 as published in (Starmans et al, 2021c). By storing all data on XNAT in a structured and standardized manner, experiments using these datasets can be easily executed at various computational resources with the same code.…”

Section: Data Statementmentioning

confidence: 99%

“…To validate our approach and evaluate its generalizability, we evaluate our framework on twelve different clinical applications using three publicly available datasets and nine inhouse datasets. To facilitate reproducibility, six of the in-house datasets with data of in total 930 patients are publicly released with this paper (Starmans et al, 2021c). To further facilitate reproducibility, we have made the software implementation of our method, and the code to perform our experiments on all datasets open-source (Starmans et al, 2018a;Starmans, 2021).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Reproducible radiomics through automated machine learning validated on twelve clinical applications

Starmans¹,

Voort²,

Phil³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Radiomics uses quantitative medical imaging features to predict clinical outcomes. While many radiomics methods have been described in the literature, these are generally designed for a single application. The aim of this study is to generalize radiomics across applications by proposing a framework to automatically construct and optimize the radiomics workflow per application. To this end, we formulate radiomics as a modular workflow, consisting of several components: image and segmentation preprocessing, feature extraction, feature and sample preprocessing, and machine learning. For each component, a collection of common algorithms is included. To optimize the workflow per application, we employ automated machine learning using a random search and ensembling. We evaluate our method in twelve different clinical applications, resulting in the following area under the curves: 1) liposarcoma (0.83); 2) desmoid-type fibromatosis (0.82); 3) primary liver tumors (0.81); 4) gastrointestinal stromal tumors (0.77); 5) colorectal liver metastases (0.68); 6) melanoma metastases (0.51); 7) hepatocellular carcinoma (0.75); 8) mesenteric fibrosis (0.81); 9) prostate cancer (0.72); 10) glioma (0.70); 11) Alzheimer's disease (0.87); and 12) head and neck cancer (0.84). Concluding, our method fully automatically constructs and optimizes the radiomics workflow, thereby streamlining the search for radiomics biomarkers in new applications. To facilitate reproducibility and future research, we publicly release six datasets, the software implementation of our framework (open-source), and the code to reproduce this study.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Evaluation Of Default Configuration On Twelve Different Clin...mentioning

confidence: 99%

Section: Evaluation Of Default Configuration On Twelve Different Clin...mentioning

confidence: 99%

Section: Data Statementmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Reproducible radiomics through automated machine learning validated on twelve clinical applications

Starmans¹,

Voort²,

Phil³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

RPTK: The Role of Feature Computation on Prediction Performance

Bohn,

Heidt,

Almeida

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Minimally interactive segmentation of soft-tissue tumors on CT and MRI using deep learning

Spaanderman,

Starmans,

van Erp

et al. 2024

Eur Radiol

View full text Add to dashboard Cite

Background Segmentations are crucial in medical imaging for morphological, volumetric, and radiomics biomarkers. Manual segmentation is accurate but not feasible in clinical workflow, while automatic segmentation generally performs sub-par. Purpose To develop a minimally interactive deep learning-based segmentation method for soft-tissue tumors (STTs) on CT and MRI. Material and methods The interactive method requires the user to click six points near the tumor’s extreme boundaries in the image. These six points are transformed into a distance map and serve, with the image, as input for a convolutional neural network. A multi-center public dataset with 514 patients and nine STT phenotypes in seven anatomical locations, with CT or T1-weighted MRI, was used for training and internal validation. For external validation, another public dataset was employed, which included five unseen STT phenotypes in extremities on CT, T1-weighted MRI, and T2-weighted fat-saturated (FS) MRI. Results Internal validation resulted in a dice similarity coefficient (DSC) of 0.85 ± 0.11 (mean ± standard deviation) for CT and 0.84 ± 0.12 for T1-weighted MRI. External validation resulted in DSCs of 0.81 ± 0.08 for CT, 0.84 ± 0.09 for T1-weighted MRI, and 0.88 ± 0.08 for T2-weighted FS MRI. Volumetric measurements showed consistent replication with low error internally (volume: 1 ± 28 mm3, r = 0.99; diameter: − 6 ± 14 mm, r = 0.90) and externally (volume: − 7 ± 23 mm3, r = 0.96; diameter: − 3 ± 6 mm, r = 0.99). Interactive segmentation time was considerably shorter (CT: 364 s, T1-weighted MRI: 258s) than manual segmentation (CT: 1639s, T1-weighted MRI: 1895s). Conclusion The minimally interactive segmentation method effectively segments STT phenotypes on CT and MRI, with robust generalization to unseen phenotypes and imaging modalities. Key Points QuestionCan this deep learning-based method segment soft-tissue tumors faster than can be done manually and more accurately than other automatic methods? Findings The minimally interactive segmentation method achieved accurate segmentation results in internal and external validation, and generalized well across soft-tissue tumor phenotypes and imaging modalities. Clinical relevance This minimally interactive deep learning-based segmentation method could reduce the burden of manual segmentation, facilitate the integration of imaging-based biomarkers (e.g., radiomics) into clinical practice, and provide a fast, semi-automatic solution for volume and diameter measurements (e.g., RECIST).

show abstract

The WORC database: MRI and CT scans, segmentations, and clinical labels for 930 patients from six radiomics studies

Cited by 15 publications

References 19 publications

Reproducible radiomics through automated machine learning validated on twelve clinical applications

Reproducible radiomics through automated machine learning validated on twelve clinical applications

RPTK: The Role of Feature Computation on Prediction Performance

Minimally interactive segmentation of soft-tissue tumors on CT and MRI using deep learning

Contact Info

Product

Resources

About