aEYE: A deep learning system for video nystagmus detection

Abstract

BackgroundNystagmus identification and interpretation is challenging for non-experts who lack specific training in neuro-ophthalmology or neuro-otology. This challenge is magnified when the task is performed via telemedicine. Deep learning models have not been heavily studied in video-based eye movement detection.MethodsWe developed, trained, and validated a deep-learning system (aEYE) to classify video recordings as normal or bearing at least two consecutive beats of nystagmus. The videos were retrospectively collected from a subset of the monocular (right eye) video-oculography (VOG) recording used in the Acute Video-oculography for Vertigo in Emergency Rooms for Rapid Triage (AVERT) clinical trial (#NCT02483429). Our model was derived from a preliminary dataset representing about 10% of the total AVERT videos (n = 435). The videos were trimmed into 10-sec clips sampled at 60 Hz with a resolution of 240 × 320 pixels. We then created 8 variations of the videos by altering the sampling rates (i.e., 30 Hz and 15 Hz) and image resolution (i.e., 60 × 80 pixels and 15 × 20 pixels). The dataset was labeled as “nystagmus” or “no nystagmus” by one expert provider. We then used a filtered image-based motion classification approach to develop aEYE. The model's performance at detecting nystagmus was calculated by using the area under the receiver-operating characteristic curve (AUROC), sensitivity, specificity, and accuracy.ResultsAn ensemble between the ResNet-soft voting and the VGG-hard voting models had the best performing metrics. The AUROC, sensitivity, specificity, and accuracy were 0.86, 88.4, 74.2, and 82.7%, respectively. Our validated folds had an average AUROC, sensitivity, specificity, and accuracy of 0.86, 80.3, 80.9, and 80.4%, respectively. Models created from the compressed videos decreased in accuracy as image sampling rate decreased from 60 Hz to 15 Hz. There was only minimal change in the accuracy of nystagmus detection when decreasing image resolution and keeping sampling rate constant.ConclusionDeep learning is useful in detecting nystagmus in 60 Hz video recordings as well as videos with lower image resolutions and sampling rates, making it a potentially useful tool to aid future automated eye-movement enabled neurologic diagnosis.

Advances in the diagnosis and management of acute vertigo

“…Early proof of concept results are very promising. 45,46 ‘Deep learning’ models would allow nystagmus identification and interpretation, 47 but have some way to go before being commercially available.…”

Smartphone-Assisted Medical Care for Vestibular Dysfunction as a Telehealth Strategy for Digital Therapy Beyond COVID-19: Scoping Review

“…After the removal of duplicates, the titles and abstracts of 289 publications were screened. Based on the inclusion and exclusion criteria, the full texts of 38 publications were read, 18 publications were excluded, and 20 publications were included in this review (Figure 1, Multimedia Appendix 1 [3,[8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26]). Among the 20 studies that reported the use of a device or app, 2 were related to anamnesis and subjective symptoms [3,8], 12 were related to objective examination [8][9][10][11][12][13][14][15][16][17][18][19], 7 were related to remote diagnosis [9,11,15,16,18,20], and 7 were related to treatment and rehabilitation [20][21][22][23][24]…”

Feasibility of video-based real-time nystagmus tracking: a lightweight deep learning model approach using ocular object segmentation

“…However, we have reached the conclusion that any model that seeks to address these problems must be lightweight and patient-centric design to facilitate efficient diagnostic workflow. Recent studies have utilized webcams ( 8 ) and smartphones ( 9 ) as well as video-oculographic devices ( 10 ) for detection of nystagmus. Thus, we have developed a new technology capable of continuously tracking eye movements by recognizing regions of the ocular.…”