Interrater reliability in visual identification of interictal high‐frequency oscillations on electrocorticography and scalp EEG

Abstract

SummaryHigh‐frequency oscillations (HFOs), including ripples (Rs) and fast ripples (FRs), are promising biomarkers of epileptogenesis, but their clinical utility is limited by the lack of a standardized approach to identification. We set out to determine whether electroencephalographers experienced in HFO analysis can reliably identify and quantify interictal HFOs. Two blinded raters independently reviewed 10 intraoperative electrocorticography (ECoG) samples from epilepsy surgery cases, and 10 scalp EEG samples from epilepsy monitoring unit evaluations. HFOs were visually marked using bandpass filters (R, 80–250 Hz; FR, 250–500 Hz) with a sampling frequency of 2,000 Hz. There was agreement as to the presence or absence of epileptiform discharges (EDs), Rs, and FRs, in 17, 18, and 18 cases, respectively. Interrater reliability (IRR) was favorable with κ = 0.70, 0.80, and 0.80, respectively, and similar for ECoG and scalp electroencephalography (EEG). Furthermore, interclass correlation for rates of Rs (0.99, 95% confidence interval [CI] 0.96–0.99) and FRs (0.77, 95% CI 0.41–0.91) were superior in comparison to EDs (0.37, 95% CI −0.60 to 0.75). Our data suggest that HFO identification and quantification are reliable among experienced electroencephalographers. Our findings support the reliability of utilizing HFO data in both research and clinical arenas.

Scalp EEG high frequency oscillations as a biomarker of treatment response in epileptic encephalopathy with continuous spike-and-wave during sleep (CSWS)

“…challenging and jeopardize its interrater reliability if raters have variable experience [43]. Our interrater reliability of visual HFOs analysis was generally favorable (our lowest kappa value was 0.65) and compatible to previous results from experienced HFOs readers [40]. The existence of physiologic HFOs has also been described and differentiating physiologic from pathologic HFOs can be challenging in intracranial recording [44,45].…”

“…Due to low signal-to-noise ratio, some of the events were difficult to be differentiated from artifacts and deemed "undetermined". This seems an inherent limitation of scalp HFOs analysis [40].…”

Refining epileptogenic high-frequency oscillations using deep learning: a reverse engineering approach

“…This work indicates that our deep learning approach can overcome issues including poor inter-rater reliability of HFO classification among human experts and their time constraints of analysis. Although our interrater reliability in this study was favorable as previously reported, 50 we expect the agreement will likely diminish when raters are from different institutions or have different experience levels. 15, 23 Using HFO analysis, we may identify brain tissue that needs to be removed during surgery without human experts’ effort if an algorithm was trained from large enough EEG data from patients with known post-operative outcomes.…”

Intracranial EEG Biomarkers for Seizure Lateralization in Rapidly-Bisynchronous Epilepsy After Laser Corpus Callosotomy

“…If the values for HFO detection are not established and fixed, it will remain impossible to translate techniques relying on the automatic detection of HFOs to any patient population; this is a major and pressing limitation of most studies involving HFO analysis. With this understanding, however, the inter-rater reliability and generalizability of even manual detection of HFOs has been questioned, (51, 52) [but, also, (53)] so this is not a concern limited to algorithmic methods, but may represent a constraint related to problem definition. Furthermore, while we note that the limitation of a 512 Hz sampling rate may have been insufficient to accurately measure higher-frequency phenomena, the 200 Hz 3-dB cutoff on our anti-aliasing filter is still considered in the middle of the "ripple" band (54).…”