Crowdsourcing seizure detection: algorithm development and validation on human implanted device recordings

Abstract

There exist significant clinical and basic research needs for accurate, automated seizure detection algorithms. These algorithms have translational potential in responsive neurostimulation devices and in automatic parsing of continuous intracranial electroencephalography data. An important barrier to developing accurate, validated algorithms for seizure detection is limited access to high-quality, expertly annotated seizure data from prolonged recordings. To overcome this, we hosted a kaggle.com competition to crowdsource the development of seizure detection algorithms using intracranial electroencephalography from canines and humans with epilepsy. The top three performing algorithms from the contest were then validated on out-of-sample patient data including standard clinical data and continuous ambulatory human data obtained over several years using the implantable NeuroVista seizure advisory system. Two hundred teams of data scientists from all over the world participated in the kaggle.com competition. The top performing teams submitted highly accurate algorithms with consistent performance in the out-of-sample validation study. The performance of these seizure detection algorithms, achieved using freely available code and data, sets a new reproducible benchmark for personalized seizure detection. We have also shared a 'plug and play' pipeline to allow other researchers to easily use these algorithms on their own datasets. The success of this competition demonstrates how sharing code and high quality data results in the creation of powerful translational tools with significant potential to impact patient care.

Automated Seizure Detection Tailored Towards Portable Patient Care Using A Novel, Computationally Efficient Method of Analysis

“…This novel method of seizure analysis achieved an accuracy rate of approximately 98.51%, comparable to that achieved by other algorithms (Saini and Dutta, 2017). While other implemented algorithms have used various high-level, computationally expensive methods of data transformation and feature extraction such as Fourier transformations and chaos theory (Baldassano et al, 2017;Saini and Dutta, 2017), the success of our novel model shows the potential of simple, broad trends of condensed EEG signals in classification. Specifically, our algorithm uses a small fraction of data points.…”

“…However, despite our novel methodology, our experiment still suffers from the same limitations as others: size of data that were tested. Our experiment, like other experiments (Baldassano et al, 2017;Liu et al, 2012;Saini and Dutta, 2017;Zhang and Chen, 2017), only used a fraction of the total amount of annotated data available. In addition, our experiment suffered from another common limitation to experimentation in the composition of our data.…”

“…Analysis of this EEG data is extremely burdensome and labor-intensive, often involving manual visual scanning of large EEG records by trained personnel. Computer-aided detection systems have been explored since the early 1970s (Baldassano et al, 2017;Saini and Dutta, 2017). Lately, many automated detection systems based on machine learning algorithms have been proposed (Feng et al, 2018;Liu et al, 2012;Saini and Dutta, 2017;Zhang and Chen, 2017).…”

Circadian Profile and Seizure Forecasting: Still Cloudy but with Chance for Sunshine

“…Machine learning, which itself had stagnated for long parts of the past half century (4), has more recently made remarkable progress, with some of the most advanced systems ironically mimicking the fundamental functions of human brain networks (5). As an example, the robustness of the most sophisticated algorithms has been demonstrated by the success of the recent kaggle.com competition sponsored by the National Institutes of Health and the American Epilepsy Society in which highly accurate algorithms for seizure detection in intracranial recordings were developed through a relatively inexpensive crowdsourcing infrastructure (6). It seems reasonable to expect clinically useful systems to be developed if a large It is now established that epilepsy is characterized by periodic dynamics that increase seizure likelihood at certain times of day, and which are highly patient-specific.…”

Computer-assisted EEG diagnostic review for idiopathic generalized epilepsy

“…The advent of machine learning and artificial intelligence has sparked a new wave of interest in developing generalizable algorithms that can be trained to recognise epileptic activity in EEG data. The seizure detection problem has even reached mainstream data science, with recent Kaggle competitions attracting thousands of entrants whose submissions covered a range of machine learning methods from deep convolutional neural networks to extreme gradient boosting [8][9][10]. A recent review of seizure detection from scalp EEG reported good performance from available algorithms, with sensitivities between 75% and 90% and false positive rates of between 0.1 and 5 per hour [6].…”