Development and Validation of a Deep Learning System to Detect Glaucomatous Optic Neuropathy Using Fundus Photographs

Abstract

IMPORTANCE A deep learning system (DLS) that could automatically detect glaucomatous optic neuropathy (GON) with high sensitivity and specificity could expedite screening for GON.OBJECTIVE To establish a DLS for detection of GON using retinal fundus images and glaucoma diagnosis with convoluted neural networks (GD-CNN) that has the ability to be generalized across populations. DESIGN, SETTING, AND PARTICIPANTSIn this cross-sectional study, a DLS for the classification of GON was developed for automated classification of GON using retinal fundus images obtained from the Chinese Glaucoma Study Alliance, the Handan Eye Study, and online databases. The researchers selected 241 032 images were selected as the training data set. The images were entered into the databases on June 9, 2009, obtained on July 11, 2018, and analyses were performed on December 15, 2018. The generalization of the DLS was tested in several validation data sets, which allowed assessment of the DLS in a clinical setting without exclusions, testing against variable image quality based on fundus photographs obtained from websites, evaluation in a population-based study that reflects a natural distribution of patients with glaucoma within the cohort and an additive data set that has a diverse ethnic distribution. An online learning system was established to transfer the trained and validated DLS to generalize the results with fundus images from new sources. To better understand the DLS decision-making process, a prediction visualization test was performed that identified regions of the fundus images utilized by the DLS for diagnosis. EXPOSURES Use of a deep learning system. MAIN OUTCOMES AND MEASURES Area under the receiver operating characteristics curve (AUC), sensitivity and specificity for DLS with reference to professional graders. RESULTS From a total of 274 413 fundus images initially obtained from CGSA, 269 601 images passed initial image quality review and were graded for GON. A total of 241 032 images (definite GON 29 865 [12.4%], probable GON 11 046 [4.6%], unlikely GON 200 121 [83%]) from 68 013 patients were selected using random sampling to train the GD-CNN model. Validation and evaluation of the GD-CNN model was assessed using the remaining 28 569 images from CGSA. The AUC of the GD-CNN model in primary local validation data sets was 0.996 (95% CI, 0.995-0.998), with sensitivity of 96.2% and specificity of 97.7%. The most common reason for both false-negative and false-positive grading by GD-CNN (51 of 119 [46.3%] and 191 of 588 [32.3%]) and manual grading (50 of 113 [44.2%] and 183 of 538 [34.0%]) was pathologic or high myopia.CONCLUSIONS AND RELEVANCE Application of GD-CNN to fundus images from different settings and varying image quality demonstrated a high sensitivity, specificity, and generalizability for detecting GON. These findings suggest that automated DLS could enhance current screening programs in a cost-effective and time-efficient manner.

Health Economic and Safety Considerations for Artificial Intelligence Applications in Diabetic Retinopathy Screening

“…6 AI-based solutions to screen for other major eye diseases, such as glaucoma and age-related macular degeneration (AMD), have also been developed. [80][81][82] However population screening for these conditions are not yet widely accepted due to inconclusive evidence based on HEA, 83 and clinically acceptable screening performance may vary for these conditions. That being said, incorporation of AI-based solutions may lower manpower costs and help make population screening for these conditions more affordable.…”

Machine learning applied to retinal image processing for glaucoma detection: review and perspective

“…Moreover, the development of a new database represented another contribution of the authors [58]. Correspondingly, Liu et al [72] developed a large-scale database of fundus images for glaucoma diagnosis (FIGD database). As for Chai et al [68], their work took advantage of both deep learning models and domain knowledge.…”

“…The most efficient results were achieved by means of cross-validation. 19:20 In 2019 as well, Liu et al [72] established a large-scale database of fundus images for glaucoma diagnosis, also known as the FIGD, and developed convoluted neural networks (GD-CNN) for automatically detecting GON. The network architecture was based on the ResNet [73].…”

Effects of Study Population, Labeling and Training on Glaucoma Detection Using Deep Learning Algorithms

“…To address this issue, an increasing number of studies include an independent testing data from other populations and geographic regions. 7,8,14 Many studies do not, however, report disease severity or other important clinical (e.g., myopia) and demographic variables (e.g., race). 7,8,13,14 Because these variables can substantially impact diagnostic accuracy, it is extremely important to report model performance as in stratified analysis by these covariates.…”