Are We There Yet? The Value of Deep Learning in a Multicenter Setting for Response Prediction of Locally Advanced Rectal Cancer to Neoadjuvant Chemoradiotherapy

Abstract

This retrospective study aims to evaluate the generalizability of a promising state-of-the-art multitask deep learning (DL) model for predicting the response of locally advanced rectal cancer (LARC) to neoadjuvant chemoradiotherapy (nCRT) using a multicenter dataset. To this end, we retrained and validated a Siamese network with two U-Nets joined at multiple layers using pre- and post-therapeutic T2-weighted (T2w), diffusion-weighted (DW) images and apparent diffusion coefficient (ADC) maps of 83 LARC patients acquired under study conditions at four different medical centers. To assess the predictive performance of the model, the trained network was then applied to an external clinical routine dataset of 46 LARC patients imaged without study conditions. The training and test datasets differed significantly in terms of their composition, e.g., T-/N-staging, the time interval between initial staging/nCRT/re-staging and surgery, as well as with respect to acquisition parameters, such as resolution, echo/repetition time, flip angle and field strength. We found that even after dedicated data pre-processing, the predictive performance dropped significantly in this multicenter setting compared to a previously published single- or two-center setting. Testing the network on the external clinical routine dataset yielded an area under the receiver operating characteristic curve of 0.54 (95% confidence interval [CI]: 0.41, 0.65), when using only pre- and post-therapeutic T2w images as input, and 0.60 (95% CI: 0.48, 0.71), when using the combination of pre- and post-therapeutic T2w, DW images, and ADC maps as input. Our study highlights the importance of data quality and harmonization in clinical trials using machine learning. Only in a joint, cross-center effort, involving a multidisciplinary team can we generate large enough curated and annotated datasets and develop the necessary pre-processing pipelines for data harmonization to successfully apply DL models clinically.

Normalization Strategies in Multi-Center Radiomics Abdominal MRI: Systematic Review and Meta-Analyses

“…All included papers have been published since 2015, with an increase from 2019. The most studied organs are the prostate [3] , [19] , [20] , [21] , [22] , [23] , [24] , [25] , [26] , [27] , [28] , [29] , [30] , [31] , [32] , [33] , [34] , [35] , [36] , [37] , [38] , [39] , [40] (S-Table I), female pelvis [4] , [41] , [42] , [43] , [44] , [45] , [46] , [47] , [48] , [49] , [50] , [51] , [52] (S-Table II), and rectum [5] , [53] , [54] , [55] , [56] , [57] , [58] , [59] , [60] , [61] , [62] , [63] , [64] , [65] , [66] , [67] , [68] , [69] (S-Table III), while a lower number of publications were related to the liver [6] , [70] , [71] , [72] , [73] , [74] , [75] (S-Table IV), and a miscellaneous group of organs including kidney [76] , [77] , …”

“…All included papers have been published since 2015, with an increase from 2019. The most studied organs are the prostate [3], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40] (S-Table I), female pelvis [4], [41], [42], [43], [44], [45], [46], [47], [48], [49], [50], [51], [52] (S-Table II), and rectum [5], [53], [54], [55], [56], [57], [58], [59], [60], [61], [62], [63],…”

Radiomics workflow definition & challenges - German priority program 2177 consensus statement on clinically applied radiomics

“…The latter is covered only briefly. These ML-based approaches are only emerging and therefore, their role in a radiomics workflow is not settled yet [ 33 , 34 ]. They might replace individual aspects of our consensus or, in the case of an end-to-end approach, even entire sequences of workflow phases.…”

An automated deep learning pipeline for EMVI classification and response prediction of rectal cancer using baseline MRI: a multi-centre study

“…The multi-task model utilises both pre and post-treatment multiparametric MRI (DWI, T2W, T1W, T1-weighted with contrast-enhancement (T1W + C)), achieving an AUC of 0.95 in two independent cohorts. However, the same model was trained by Wichtman et al 24 in a multi-centre (4 centres) scenario. Their model showed an AUC of 0.60 when using the combination of pre and post-therapeutic T2W, DWI, and ADC maps as input.…”

“…Their model showed an AUC of 0.60 when using the combination of pre and post-therapeutic T2W, DWI, and ADC maps as input. Wichtmann et al 24 demonstrated the current challenge of constructing deep learning models using multi-institutional medical data. Data from different origins can contain significant variations based on specific parametrisation, creating a domain shift problem observed in multiple medical imaging modalities 25 , 26 .…”