Framework for Personalized Prediction of Treatment Response in Relapsing-Remitting Multiple Sclerosis: A Replication Study in Independent Data

Abstract

Background Individualizing and optimizing treatment of relapsing-remitting multiple sclerosis patients is a challenging problem, which would benefit from a clinically valid decision support. Stühler et al. presented black box models for this aim which were developed and internally evaluated in a German registry but lacked external validation. Methods In patients from the French OFSEP registry, we independently built and validated models predicting being free of relapse and free of confirmed disability progression (CDP), following the methodological roadmap and predictors reported by Stühler. Hierarchical Bayesian models were fit to predict the outcomes under 6 disease-modifying treatments given the individual disease course up to the moment of treatment change. Data was temporally split on 2017, and models were developed in patients treated earlier (n = 5517). Calibration curves, discrimination, and mean squared error (MSE) were assessed by external validation of models in more-recent patients (n = 3768). Standard count models were also applied and compared to the Bayesian ones. Results The performance of our temporally-validated relapse model (MSE: 0.326, C-Index: 0.639) is potentially superior to that of Stühler’s (MSE: 0.784, C-index: 0.608). Calibration plots revealed miscalibration. Our CDP model (MSE: 0.072, C-Index: 0.777) was also better than its counterpart (MSE: 0.131, C-index: 0.554). Results from standard count models were similar to the Bayesian ones. Conclusions The CDP model rebuilt and externally validated in independent data compared and strengthened the credibility of the Stühler models. Their model-building strategy was replicable.