Reliability of Graders and Comparison with an Automated Algorithm for Vertical Cup-Disc Ratio Grading in Fundus Photographs

Abstract

Introduction: We aimed to investigate the intergrader and intragrader reliability of human graders and an automated algorithm for vertical cup-disc ratio (CDR) grading in colour fundus photographs. Materials and Methods: Two-hundred fundus photographs were selected from a database of 3000 photographs of patients screened at a tertiary ophthalmology referral centre. The graders included glaucoma specialists (n = 3), general ophthalmologists (n = 2), optometrists (n = 2), family physicians (n = 2) and a novel automated algorithm (AA). In total, 2 rounds of CDR grading were held for each grader on 2 different dates, with the photographs presented in random order. The CDR values were graded as 0.1-1.0 or ungradable. The grading results of the 2 senior glaucoma specialists were used as the reference benchmarks for comparison. Results: The intraclass correlation coefficient values ranged from 0.37-0.74 and 0.47-0.97 for intergrader and intragrader reliability, respectively. There was no significant correlation between the human graders’ level of reliability and their years of experience in grading CDR (P = 0.91). The area under the curve (AUC) value of the AA was 0.847 (comparable to AUC value of 0.876 for the glaucoma specialist). Bland Altman plots demonstrated that the AA’s performance was at least comparable to a glaucoma specialist. Conclusion: The results suggest that AA is comparable to and may have more consistent performance than human graders in CDR grading of fundus photographs. This may have potential application as a screening tool to help detect asymptomatic glaucoma-suspect patients in the community. Key words: Eyes, Glaucoma, Intraocular pressure

Accuracy of vertical cup-to-disc ratio discrimination among clinical optometry trainees with different years of clinical experience

“…These findings were attributed to theoretical teaching that was primarily conducted in the first year of the residents' training. Besides, another study reported no significant correlation between VCDR's graders' reliability and their years of experience in grading [6]. In our study, the clinical optometry trainees were taught about the ways to discriminate VCDR when they were in Year 2.…”

“…CDR can be measured horizontally or vertically, giving the horizontal cup-to-disc ratio (HCDR) and vertical cup-to-disc ratio (VCDR), respectively. VCDR is usually used as an indication for glaucoma screening [1,[4][5][6]. An illustration of the ONH and VCDR is shown in Fig 1 . VCDR equal to or larger than 0.8, or a difference in VCDR between eyes by more than 0.2, is pathological [7].…”

RimNet: A Deep Neural Network Pipeline for Automated Identification of the Optic Disc Rim

“…5 It is commonly accepted and has been incorporated into eye health guidelines for optometrists and ophthalmologists. 7 , 8 The DDLS is limited as a diagnostic tool by the need for expert time to accurately grade images. Automated high-efficacy DDLS grading could offer a powerful screening method.…”

“…First, studies have shown that automated algorithms can offer more consistent and reliable grading than human graders. 8 Second, there has been a rapid advancement in image segmentation, image processing, and deep learning neural networks. In other fields, several neural networks outperformed human graders in image classification tasks.…”