In Vivo Repeatability and Multiscanner Reproducibility of MRI Radiomics Features in Patients With Monoclonal Plasma Cell Disorders

Abstract

Objectives: Despite the extensive number of publications in the field of radiomics, radiomics algorithms barely enter large-scale clinical application. Supposedly, the low external generalizability of radiomics models is one of the main reasons, which hinders the translation from research to clinical application. The objectives of this study were to investigate reproducibility of radiomics features (RFs) in vivo under variation of patient positioning, magnetic resonance imaging (MRI) sequence, and MRI scanners, and to identify a subgroup of RFs that shows acceptable reproducibility across all different acquisition scenarios.Materials and Methods: Between November 30, 2020 and February 16, 2021, 55 patients with monoclonal plasma cell disorders were included in this prospective, bi-institutional, single-vendor study. Participants underwent one reference scan at a 1.5 T MRI scanner and several retest scans: once after simple repositioning, once with a second MRI protocol, once at another 1.5 T scanner, and once at a 3 T scanner. Radiomics feature from the bone marrow of the left hip bone were extracted, both from original scans and after different image normalizations. Intraclass correlation coefficient (ICC) was used to assess RF repeatability and reproducibility. Results: Fifty-five participants (mean age, 59 ± 7 years; 36 men) were enrolled. For T1-weighted images after muscle normalization, in the simple test-retest experiment, 110 (37%) of 295 RFs showed an ICC ≥0.8: 54 (61%) of 89 first-order features (FOFs), 35 (95%) of 37 volume and shape features, and 21 (12%) of 169 texture features (TFs). When the retest was performed with different technical settings, even after muscle normalization, the number of FOF/TF with an ICC ≥0.8 declined to 58/13 for the second protocol, 29/7 for the second 1.5 T scanner, and 49/7 for the 3 T scanner, respectively. Twenty-five (28%) of the 89 FOFs and 6 (4%) of the 169 TFs from muscle-normalized T1-weighted images showed an ICC ≥0.8 throughout all repeatability and reproducibility experiments. Conclusions: In vivo, only few RFs are reproducible with different MRI sequences or different MRI scanners, even after application of a simple image normalization. Radiomics features selected by a repeatability experiment only are not necessarily suited to build radiomics models for multicenter clinical application. This study isolated a subset of RFs, which are robust to variations in MRI acquisition observed in scanners from 1 vendor, and therefore are candidates to build reproducible radiomics models for monoclonal plasma cell disorders for multicentric applications, at least when centers are equipped with scanners from this vendor.

Machine Learning Model Based on Optimized Radiomics Feature from 18F-FDG-PET/CT and Clinical Characteristics Predicts Prognosis of Multiple Myeloma: A Preliminary Study

“…Radiomics features selected by a repeatability experiment only are not necessarily suited to build radiomics models for multicenter clinical application. Supposedly, one of the main reasons that hinder the translation to clinical application is the low external generalizability of radiomics models [ 29 ]. Accordingly, standardization of image acquisition or advanced calculative approaches for image normalization or RF compensation might help to improve external generalizability of radiomics prediction models.…”

Weakly Supervised Learning with Positive and Unlabeled Data for Automatic Brain Tumor Segmentation

“…The segmentation of the region of interest is a major step for this approach. While manual segmentation is possible, it is known that the inter-rater variability of such approaches has a major impact on the quality and reliability of the radiomics study [42][43][44][45]. While there are methods to control for the influence of rater variability from the segmentations [45,46], the usage of automatic segmentations is often considered better as the transfer to other clinical settings is more straightforward [47].…”

Prediction of Bone Marrow Biopsy Results From MRI in Multiple Myeloma Patients Using Deep Learning and Radiomics

“…Before feature extraction, all images were resampled to a uniform voxel spacing and normalized to the mean signal intensity of the piriformis muscle to minimize heterogeneity between data sets caused by technical variations in image acquisition. 26,28 The IBSI-conform 29 and validated software MITK Phenotyping 27 were used for radiomics feature calculation. A total of 260 radiomics features were calculated, with 91 first-order features and 169 texture features.…”