Accuracy of a Deep Learning System for Classification of Papilledema Severity on Ocular Fundus Photographs

Abstract

Objective:To evaluate the performance of a deep learning system (DLS) in classifying the severity of papilledema associated with increased intracranial pressure, on standard retinal fundus photographs.Methods:A DLS was trained to automatically classify papilledema severity in 965 patients (2103 mydriatic fundus photographs), representing a multiethnic cohort of patients with confirmed elevated intracranial pressure. Training was performed on 1052 photographs with mild/moderate papilledema (MP) and 1051 photographs with severe papilledema (SP) classified by a panel of experts. The performance of the DLS and that of three independent neuro-ophthalmologists were tested in 111 patients (214 photographs, 92 with MP and 122 with SP), by calculating the area under the receiver operating characteristics curve (AUC), accuracy, sensitivity and specificity. Kappa agreement scores between the DLS and each of the three graders and among the three graders were calculated.Results:The DLS successfully discriminated between photographs of MP and SP, with an AUC of 0.93 (95%CI: 0.89-0.96) and an accuracy, sensitivity and specificity of 87.9%, 91.8% and 86.2%, respectively. This performance was comparable with that of the three neuro-ophthalmologists (84.1%, 91.8% and 73.9%, P=0.19, P=1, P=0.09, respectively). Misclassification by the DLS was mainly observed for moderate papilledema (Frisén grade 3). Agreement scores between the DLS and the neuro-ophthalmologists’ evaluation was 0.62 (CI 95% 0.57-0.68), whereas the inter-grader agreement among the three neuro-ophthalmologists was 0.54 (CI 95% 0.47-0.62).Conclusions:Our DLS accurately classified the severity of papilledema on an independent set of mydriatic fundus photographs, achieving a comparable performance with that of independent neuro-ophthalmologists.Classification of Evidence:This study provides Class II evidence that a deep learning system using mydriatic retinal fundus photographs accurately classified the severity of papilledema associated in patients with a diagnosis of increased intracranial pressure.

A recurrent machine learning model predicts intracranial hypertension in neurointensive care patients

“… 31 A 30-min time window to an critical ICP increment was chosen to be sufficient for clinical decision-making. 32 Recent approaches try to integrate medical imaging information to predict ICP 33 , 34 or intracranial pathologies. 35 On the other hand the ICP signal itself can be used to predict ventriculitis with machine learning.…”

Artificial Intelligence Meets Neuro-Ophthalmology

“…The DLS yielded an AUC of 0.93 (95% CI: 0.89–0.96), an accuracy of 87.9%, a sensitivity of 91.8%, and a specificity of 82.6% in classifying papilledema as mild/ moderate versus severe. This classification performance was not significantly different from that of neuro-ophthalmologists 44 . It is worth noting that the majority of misclassifications occurred on photographs of papilledematous ONH with a Frisen grade 3 (14 out of 26 misclassifications).…”

“…This classification performance was not significantly different from that of neuro-ophthalmologists. 44 It is worth noting that the majority of misclassifications occurred on photographs of papilledematous ONH with a Frisen grade 3 (14 out of 26 misclassifications).…”

Real-World Translation of Artificial Intelligence in Neuro-Ophthalmology: The Challenges of Making an Artificial Intelligence System Applicable to Clinical Practice

“…Vasseneix et al (16) took a step toward transparency by describing common features in images that were misclassified by their model. A more comprehensive approach is to use a black-box explainability method such as Grad-CAM (32) that highlights image features the network was considering during classification.…”