The Validation of Deep Learning-Based Grading Model for Diabetic Retinopathy

Abstract

PurposeTo evaluate the performance of a deep learning (DL)-based artificial intelligence (AI) hierarchical diagnosis software, EyeWisdom V1 for diabetic retinopathy (DR).Materials and MethodsThe prospective study was a multicenter, double-blind, and self-controlled clinical trial. Non-dilated posterior pole fundus images were evaluated by ophthalmologists and EyeWisdom V1, respectively. The diagnosis of manual grading was considered as the gold standard. Primary evaluation index (sensitivity and specificity) and secondary evaluation index like positive predictive values (PPV), negative predictive values (NPV), etc., were calculated to evaluate the performance of EyeWisdom V1.ResultsA total of 1,089 fundus images from 630 patients were included, with a mean age of (56.52 ± 11.13) years. For any DR, the sensitivity, specificity, PPV, and NPV were 98.23% (95% CI 96.93–99.08%), 74.45% (95% CI 69.95-78.60%), 86.38% (95% CI 83.76-88.72%), and 96.23% (95% CI 93.50-98.04%), respectively; For sight-threatening DR (STDR, severe non-proliferative DR or worse), the above indicators were 80.47% (95% CI 75.07-85.14%), 97.96% (95% CI 96.75-98.81%), 92.38% (95% CI 88.07-95.50%), and 94.23% (95% CI 92.46-95.68%); For referral DR (moderate non-proliferative DR or worse), the sensitivity and specificity were 92.96% (95% CI 90.66-94.84%) and 93.32% (95% CI 90.65-95.42%), with the PPV of 94.93% (95% CI 92.89-96.53%) and the NPV of 90.78% (95% CI 87.81-93.22%). The kappa score of EyeWisdom V1 was 0.860 (0.827-0.890) with the AUC of 0.958 for referral DR.ConclusionThe EyeWisdom V1 could provide reliable DR grading and referral recommendation based on the fundus images of diabetics.

Artificial Intelligence in Community-Based Diabetic Retinopathy Telemedicine Screening in Urban China: Cost-effectiveness and Cost-Utility Analyses With Real-world Data

“…The accuracy of AI-based telemedicine screening was extracted from published studies specific to the AI-assisted screening model conducted in Shanghai based on the current dominant architecture of convolutional neural networks (Multimedia Appendix 10) [25]. Briefly, the sensitivity was 80.47% (95% CI 75.07%-85.14%) and the specificity was 97.96% (95% CI 96.75%-98.81%) for STDR [25]. In our screening program, 2 experienced ophthalmologists were employed to make the diagnoses based on the retinal images.…”

Research progress on diagnosing retinal vascular diseases based on artificial intelligence and fundus images

“…Finally, the results showed that the accuracy, sensitivity, and specificity of the model on the EyePACS-1 dataset were 0.899, 0.882, and 0.913, respectively, and the accuracy, sensitivity and specificity of the model on the Messidor-2 dataset were 0.918, 0.902, and 0.930, respectively. To assist DR classification, Zhang W. F et al (2022) developed an AI classification model based on ResNet-34 and Inception-v3 and used 1,089 fundus images to train and test the model. After testing, the AUC of the model was 0.958 and the kappa score was 0.860.…”

Research progress in artificial intelligence assisted diabetic retinopathy diagnosis

“…They used the APTOS 2019 dataset to evaluate the performance of the AI model, and the final results showed that the DR classification model's accuracy was as high as 0.98. Similarly, Zhang et al [60] constructed an AI hierarchical model by ResNet-34 and Inception-V3 networks, which can automatically divide DR into five levels: no DR, mild DR, moderate DR, severe DR, PDR. In addition, they collected 1089 color fundus photographs to evaluate the performance of the AI model.…”