Development and Validation of Forecasting Next Reported Seizure Using e‐Diaries

Abstract

Objective There are no validated methods for predicting the timing of seizures. Using machine learning, we sought to forecast 24‐hour risk of self‐reported seizure from e‐diaries. Methods Data from 5,419 patients on http://SeizureTracker.com (including seizure count, type, and duration) were split into training (3,806 patients/1,665,215 patient‐days) and testing (1,613 patients/549,588 patient‐days) sets with no overlapping patients. An artificial intelligence (AI) program, consisting of recurrent networks followed by a multilayer perceptron (“deep learning” model), was trained to produce risk forecasts. Forecasts were made from a sliding window of 3‐month diary history for each day of each patient's diary. After training, the model parameters were held constant and the testing set was scored. A rate‐matched random (RMR) forecast was compared to the AI. Comparisons were made using the area under the receiver operating characteristic curve (AUC), a measure of binary discrimination performance, and the Brier score, a measure of forecast calibration. The Brier skill score (BSS) measured the improvement of the AI Brier score compared to the benchmark RMR Brier score. Confidence intervals (CIs) on performance statistics were obtained via bootstrapping. Results The AUC was 0.86 (95% CI = 0.85–0.88) for AI and 0.83 (95% CI = 0.81–0.85) for RMR, favoring AI (p < 0.001). Overall (all patients combined), BSS was 0.27 (95% CI = 0.23–0.31), also favoring AI (p < 0.001). Interpretation The AI produced a valid forecast superior to a chance forecaster, and provided meaningful forecasts in the majority of patients. Future studies will be needed to quantify the clinical value of these forecasts for patients. ANN NEUROL 2020;88:588–595

Cycles of self‐reported seizure likelihood correspond to yield of diagnostic epilepsy monitoring

“…In contrast, forecasts based on “black‐box” machine learning models cannot be projected beyond the range of the available data, and therefore are less flexible for making long‐range estimates of seizure likelihood. The increasing availability of electronic diary data has recently advanced machine learning techniques for diary‐based seizure forecasting 30,44,45 . However, it is important to bear in mind that self‐reported diaries provide a noisy, undersampled representation of the underlying true seizure likelihood.…”

“…We have shown that multiday seizure cycles can be measured noninvasively using self‐reported seizure times; and, for most people, seizure cycles measured from self‐reported events correlated with their cycles of electrographic seizures 29 . There is now growing evidence that seizure diaries can be used to generate accurate, personalized forecasts of future seizure likelihood 29–31 . Diary‐based forecasts are compelling due to their widespread availability, ease of use, and low‐cost implementation (typically available via free digital platforms or mobile apps) 32 .…”

Cycles of self-reported seizure likelihood correspond to yield of diagnostic epilepsy monitoring

“…In contrast, forecasts based on “black-box” machine learning models cannot be projected beyond the range of the available data, so are less flexible for making long-range estimates of seizure likelihood. The increasing availability of electronic diary data has recently advanced machine learning techniques for diary-based seizure forecasting 30,44,45 .…”

Can Big Data guide prognosis and clinical decisions in epilepsy?

“…Jin et al developed a neural network classifier with magnetic resonance imaging data to identify solitary focal cortical dysplasias (FCDs) in a dataset of 61 patients with type II FCDs as well as 120 controls from three different epilepsy centers and achieved a sensitivity of 73.7%, specificity of 90.0%, and area under the curve (AUC) for the receiver operating characteristic analysis of .75 52 . Goldenholz et al proposed an approach with recurrent networks and multilayer perceptron to forecast the probability of future seizures; the evaluation results achieved an AUC of .86 on the testing set of 1613 patients 53 . Abbasi and Goldenholz provided a detailed review of machine learning applications with different domains in epilepsy, such as automated seizure detection, analysis of imaging and clinical data, epilepsy localization, and prediction of medical and surgical outcomes 40 …”

“…52 Goldenholz et al proposed an approach with recurrent networks and multilayer perceptron to forecast the probability of future seizures; the evaluation results achieved an AUC of .86 on the testing set of 1613 patients. 53 Abbasi and Goldenholz provided a detailed review of machine learning applications with different domains in…”