Diagnostic accuracy of diabetic retinopathy grading by an artificial intelligence-enabled algorithm compared with a human standard for wide-field true-colour confocal scanning and standard digital retinal images

Abstract

BackgroundPhotographic diabetic retinopathy screening requires labour-intensive grading of retinal images by humans. Automated retinal image analysis software (ARIAS) could provide an alternative to human grading. We compare the performance of an ARIAS using true-colour, wide-field confocal scanning images and standard fundus images in the English National Diabetic Eye Screening Programme (NDESP) against human grading.MethodsCross-sectional study with consecutive recruitment of patients attending annual diabetic eye screening. Imaging with mydriasis was performed (two-field protocol) with the EIDON platform (CenterVue, Padua, Italy) and standard NDESP cameras. Human grading was carried out according to NDESP protocol. Images were processed by EyeArt V.2.1.0 (Eyenuk Inc, Woodland Hills, California). The reference standard for analysis was the human grade of standard NDESP images.ResultsWe included 1257 patients. Sensitivity estimates for retinopathy grades were: EIDON images; 92.27% (95% CI: 88.43% to 94.69%) for any retinopathy, 99% (95% CI: 95.35% to 100%) for vision-threatening retinopathy and 100% (95% CI: 61% to 100%) for proliferative retinopathy. For NDESP images: 92.26% (95% CI: 88.37% to 94.69%) for any retinopathy, 100% (95% CI: 99.53% to 100%) for vision-threatening retinopathy and 100% (95% CI: 61% to 100%) for proliferative retinopathy. One case of vision-threatening retinopathy (R1M1) was missed by the EyeArt when analysing the EIDON images, but identified by the human graders. The EyeArt identified all cases of vision-threatening retinopathy in the standard images.ConclusionEyeArt identified diabetic retinopathy in EIDON images with similar sensitivity to standard images in a large-scale screening programme, exceeding the sensitivity threshold recommended for a screening test. Further work to optimise the identification of ‘no retinopathy’ and to understand the differential lesion detection in the two imaging systems would enhance the use of these two innovative technologies in a diabetic retinopathy screening setting.

Evidence Based Prediction and Progression Monitoring on Retinal Images from Three Nations

“…Testing the data collected from three nations with variation of these factors showed that, as long as the quality assessment is in place in the workflow, the performance of the algorithm is consistent. This is in line with other studies, 29 , 30 where AI was evaluated in coordination with either assessing the quality and protocol adherence of images, or data imaged with mydriasis through a high-quality imaging platform.…”

In-Person Verification of Deep Learning Algorithm for Diabetic Retinopathy Screening Using Different Techniques Across Fundus Image Devices

“…Only a study by Olvera-Barrios and coauthors reported a similar sensitivity for detecting DR (92%), compared with human graders using the commercially available AI algorithm EyeArt (version 2.1.0; Eyenuk, Inc., Woodland Hills, CA). 25 In contrast, our study used separate models for Nidek and Eidon due to the poor diagnostic accuracy achieved when using the same DR screening model. The difference in the model accuracies of the two fundus cameras might result from differences in the color, image resolution, field of view, and lesion characteristics observable in their respective images.…”

Recent Advances and Clinical Application of Color Scanning Laser Ophthalmoscope

“…Recently, there has been a lot of research on artificial intelligence (AI) using fundus photographs, OCT, and OCT angiography images [36][37][38][39][40], and there is a highly likelihood that color SLO, which has high image quality with a large amount of information, will be useful for this kind of automatic diagnosis by AI. Indeed, preliminary studies using AI and color SLO have been reported [16,41,42].…”

“…[14]. In addition, a confocal light-emitting diode-based retinal imaging system (Eidon, Centervue, Padova, Italy), which is a similar concept with the color SLO, was also launched [15,16].…”