IntroductionCheckpoint inhibitor treatment has proven successful for advanced melanoma. However, a significant fraction of patients does not experience benefit from this treatment, that is also associated with potentially severe toxicity and high costs. Previous research has not yet resulted in adequate biomarkers that can predict treatment outcomes. The present work is the first to investigate the value of deep learning on computed tomography (CT) imaging of melanoma lesions for predicting checkpoint inhibitor treatment outcomes in advanced melanoma.MethodsAdult patients that were treated with first line anti-PD1 ± anti-CTLA4 therapy for unresectable stage IIIC or stage IV melanoma were retrospectively identified from ten participating centers. Up to five representative lesions were segmented volumetrically on baseline CT; a deep learning model (DLM) was trained on the corresponding volumes to predict clinical benefit, defined as stable disease for a minimum of six months, or response at any time during follow-up. Optimal hyperparameters and model types (Densenet, Efficientnet, Squeeze-Excitation ResNet, ResNeXt) were iteratively explored. The DLM was compared to a model of previously identified clinical predictors (presence of liver and brain metastasis, level of lactate dehydrogenase, performance status and number of affected organs), and a combination model consisting of both clinical predictors and the DLM.ResultsA total of 730 eligible patients with 2722 lesions were included. Rate of clinical benefit was 59.6%. The selected deep learning model was a Squeeze-Excitation ResNet with random initialization, trained with the Adam optimizer. The DLM reached an area under the receiver operating characteristic (AUROC) of 0.607 [95% CI 0.565 – 0.648]. In comparison, a model of clinical predictors reached an AUROC of 0.635 [95% CI 0.592 – 0.678]. The combination model reached an AUROC of 0.635 [95% CI 0.595 – 0.676]. None of the differences in AUROC were statistically significant. The output of the DLM was significantly correlated with four of the five input variables of the clinical model.DiscussionAlthough the DLM reached a statistically significant discriminative value, it was unable to improve over previously identified clinical predictors. The most likely cause is that the DLM learns to detect a lesion’s size and organ location, which is information that is already present in the clinical model. Given the substantial sample size and extensive hyperparameter optimization, this indicates that the predictive value of CT imaging of lesions for checkpoint inhibitor response in melanoma is likely limited. The present work shows that the assessment over known clinical predictors is an essential step for imaging-based prediction and brings important nuance to the almost exclusively positive findings in this field.