Prospective validation study of an epilepsy seizure risk system for outpatient evaluation

Abstract

Objective: We conducted clinical testing of an automated Bayesian machine learning algorithm (Epilepsy Seizure Assessment Tool [EpiSAT]) for outpatient seizure risk assessment using seizure counting data, and validated performance against specialized epilepsy clinician experts. Methods: We conducted a prospective longitudinal study of EpiSAT performance against 24 specialized clinician experts at three tertiary referral epilepsy centers in the United States. Accuracy, interrater reliability, and intra-rater reliability of EpiSAT for correctly identifying changes in seizure risk (improvements, worsening, or no change) were evaluated using 120 seizures from four synthetic seizure diaries (seizure risk known) and 120 seizures from four real seizure diaries (seizure risk unknown). The proportion of observed agreement between EpiSAT and clinicians was evaluated to assess compatibility of EpiSAT with clinical decision patterns by epilepsy experts. Results: EpiSAT exhibited substantial observed agreement (75.4%) with clinicians for assessing seizure risk. The mean accuracy of epilepsy providers for correctly assessing seizure risk was 74.7%. EpiSAT accurately identified seizure risk in 87.5% of seizure diary entries, corresponding to a significant improvement of 17.4% (P = .002). Clinicians exhibited low-to-moderate interrater reliability for seizure risk assessment (Krippendorff's α = 0.46) with good intrarater reliability across a 4-to 12-week evaluation period (Scott's π = 0.89). Significance: These results validate the ability of EpiSAT to yield objective clinical recommendations on seizure risk which follow decision patterns similar to those from specialized epilepsy providers, but with improved accuracy and reproducibility. This algorithm may serve as a useful clinical decision support system for quantitative analysis of clinical seizure frequency in clinical epilepsy practice.

Seizure forecasting and cyclic control of seizures

“…Epilepsy seizure diaries have also shown promise as a source of diverse clinical insights, 45 including as a measure of seizure risk 100 . Despite well‐documented inaccuracies and limitations inherent to seizure diaries, 101 self‐reported events remain the standard data source for medical practice and clinical trials in epilepsy.…”

Can Big Data guide prognosis and clinical decisions in epilepsy?

“…It can automatically identify the probability of changes in the frequency of patient-reported clinical seizures; the probability indicates that the susceptibility to seizures has worsened or improved. 28 EpiSAT accurately identified seizure risk in 87.5% of seizure diary entries and achieved 75.4% agreement in decision patterns with clinicians. Imaging efforts for CDSSs are likely to make inroads too, and although these are still at an early stage, there is significant promise of scale.…”

“…EpiSAT is a quantitative tool for seizure risk assessment. It can automatically identify the probability of changes in the frequency of patient‐reported clinical seizures; the probability indicates that the susceptibility to seizures has worsened or improved 28 . EpiSAT accurately identified seizure risk in 87.5% of seizure diary entries and achieved 75.4% agreement in decision patterns with clinicians.…”

Ranking Epilepsy Risk Factors Using Machine Learning

“…These methods are actively used in epileptology, for example, to predict the pharmacoresistant epilepsy [16], surgical treatment outcomes [17], to determine the epileptogenic zone [18] and functional neural systems in patients with epilepsy [19], develop new classi cation approaches [20] [21], and perform other tasks [22]. Machine learning models are actively used in decision support systems to treat patients with various forms of epilepsy [23], [24]. At the same time, classical statistical methods of analysis are usually used to identify factors associated with the development of epilepsy within a typical case-control study design.…”