Development and Validation of a Deep Learning Model for Predicting Treatment Response in Patients With Newly Diagnosed Epilepsy

Abstract

ImportanceSelection of antiseizure medications (ASMs) for epilepsy remains largely a trial-and-error approach. Under this approach, many patients have to endure sequential trials of ineffective treatments until the “right drugs” are prescribed.ObjectiveTo develop and validate a deep learning model using readily available clinical information to predict treatment success with the first ASM for individual patients.Design, Setting, and ParticipantsThis cohort study developed and validated a prognostic model. Patients were treated between 1982 and 2020. All patients were followed up for a minimum of 1 year or until failure of the first ASM. A total of 2404 adults with epilepsy newly treated at specialist clinics in Scotland, Malaysia, Australia, and China between 1982 and 2020 were considered for inclusion, of whom 606 (25.2%) were excluded from the final cohort because of missing information in 1 or more variables.ExposuresOne of 7 antiseizure medications.Main Outcomes and MeasuresWith the use of the transformer model architecture on 16 clinical factors and ASM information, this cohort study first pooled all cohorts for model training and testing. The model was trained again using the largest cohort and externally validated on the other 4 cohorts. The area under the receiver operating characteristic curve (AUROC), weighted balanced accuracy, sensitivity, and specificity of the model were all assessed for predicting treatment success based on the optimal probability cutoff. Treatment success was defined as complete seizure freedom for the first year of treatment while taking the first ASM. Performance of the transformer model was compared with other machine learning models.ResultsThe final pooled cohort included 1798 adults (54.5% female; median age, 34 years [IQR, 24-50 years]). The transformer model that was trained using the pooled cohort had an AUROC of 0.65 (95% CI, 0.63-0.67) and a weighted balanced accuracy of 0.62 (95% CI, 0.60-0.64) on the test set. The model that was trained using the largest cohort only had AUROCs ranging from 0.52 to 0.60 and a weighted balanced accuracy ranging from 0.51 to 0.62 in the external validation cohorts. Number of pretreatment seizures, presence of psychiatric disorders, electroencephalography, and brain imaging findings were the most important clinical variables for predicted outcomes in both models. The transformer model that was developed using the pooled cohort outperformed 2 of the 5 other models tested in terms of AUROC.Conclusions and RelevanceIn this cohort study, a deep learning model showed the feasibility of personalized prediction of response to ASMs based on clinical information. With improvement of performance, such as by incorporating genetic and imaging data, this model may potentially assist clinicians in selecting the right drug at the first trial.

Choosing the Best Antiseizure Medication—Can Artificial Intelligence Help?

“…4 These are significant advantages that use the power of deep learning to achieve better predictions for data in which long sequences of data are present, which is especially pertinent in ASM histories. Given that Hakeem and colleagues' application of a transformer model in this study 1 does not involve sequence data, it will be of great interest in future studies to reassess how the transformer model performs compared with other traditional ML methods when applied to longitudinal sequences of input data, considering this is where transformers have been found to excel.…”

“…Indeed, there are also a few generalizability limitations that remain owing to imbalance in the data set, as acknowledged by Hakeem and colleagues. 1 For example, despite the large size of the patient cohorts in this study, only 7 mostly older-generation ASMs were analyzed. In addition, the data set analyzed has a low representation of people with generalized epilepsy or intellectual disability, groups that often have generalized or multifocal seizures that are not surgically amenable 8 and who stand to benefit greatly from earlier ASM optimization.…”

Deep Thoughts—Predicting Initial Treatment Response in Newly Diagnosed Epilepsy

“…But consider this striking statistic stated differently: 71% of initial treatment is unsuccessful. 1 Now, “unsuccessful” is a grab-bag term conflating inefficacy, nonadherence, and intolerability. Regardless, that’s a problem.…”

“…Hakeem et al tackled this question. 1 In background, they point to a precursor model 2 developed on claims data predicting optimal first ASM selection, with modest discrimination. While claims provide sheer size, they imperfectly measure some key predictors and outcomes (e.g., seizures, electroencephalogram (EEG)/imaging test results, drug discontinuation).…”

Development and Validation of a Machine Learning Model to Predict Prognosis in HIV-Negative Cryptococcal Meningitis Patients: A Multicentre Retrospective Study

“…A large number of studies have utilized machine learning to assist clinical decision-making, screen high-risk groups early, and realize personalized treatment [7][8][9][10]. And Machine learning applications have been used to help predict the prognosis of some central nervous system diseases such as multiple sclerosis [11], ischemic stroke [12], and epilepsy [13], and they have shown good prediction performance. Thus far there have been few studies that examined speci c machine learning algorithms in predicting prognosis in HIV-negative CM patients.…”