The SEE Study: Safety, Efficacy, and Equity of Implementing Autonomous Artificial Intelligence for Diagnosing Diabetic Retinopathy in Youth

Abstract

Diabetic retinopathy (DR) is a leading cause of vision loss worldwide. Screening for DR is recommended in children and adolescents, but adherence is poor. Recently, autonomous artificial intelligence (AI) systems have been developed for early detection of DR and have been included in the American Diabetes Association's guidelines for screening in adults. We sought to determine the diagnostic efficacy of autonomous AI for the diabetic eye exam in youth with diabetes. RESEARCH DESIGN AND METHODSIn this prospective study, point-of-care diabetic eye exam was implemented using a nonmydriatic fundus camera with an autonomous AI system for detection of DR in a multidisciplinary pediatric diabetes center. Sensitivity, specificity, and diagnosability of AI was compared with consensus grading by retinal specialists, who were masked to AI output. Adherence to screening guidelines was measured before and after AI implementation. RESULTSThree hundred ten youth with diabetes aged 5-21 years were included, of whom 4.2% had DR. Diagnosability of AI was 97.5% (302 of 310). The sensitivity and specificity of AI to detect more-than-mild DR was 85.7% (95% CI 42.1-99.6%) and 79.3% (74.3-83.8%), respectively, compared with the reference standard as defined by retina specialists. Adherence improved from 49% to 95% after AI implementation. CONCLUSIONSUse of a nonmydriatic fundus camera with autonomous AI was safe and effective for the diabetic eye exam in youth in our study. Adherence to screening guidelines improved with AI implementation. As the prevalence of diabetes increases in youth and adherence to screening guidelines remains suboptimal, effective strategies for diabetic eye exams in this population are needed.Diabetes is a significant public health problem worldwide, and in the U.S., it affects almost 30 million people (1). In youth, diabetes is one of the most common chronic childhood diseases, with an incidence that has been increasing over the past decade for both type 1 diabetes (T1D) and type 2 diabetes (T2D) (2). Children with diabetes are at risk for diabetes-related complications, including diabetic retinopathy (DR), which

Clinical and Demographic Factors Associated With Diabetic Retinopathy Among Young Patients With Diabetes

“…Methods for data collection have been described previously. 26 In brief, point-of-care DR screening was performed during routine care visits between December 3, 2018, and November 1, 2019, as part of a prospective study of patients aged 5 to 21 years with type 1 or type 2 diabetes. Two color images per eye (1 macula-centered and 1 disccentered) were obtained using a nonmydriatic fundus camera (TRC NW400; Topcon Corp) and graded independently by 2 retina specialists (one of whom was R.C.)…”

“…20 The use of fundus photography located in pediatric offices rather than ophthalmologic clinics in our study was likely helpful to our ability to form a large cohort of patients because increased ease of access to DR screenings increases adherence to guidelines. 26,53…”

“…20 Although a dilated and comprehensive eye examination by an ophthalmologist remains the standard of care for retinal screening, fundus photography with or without artificial intelligence-based techniques for point-of-care detection of DR serves as an accurate screening tool for DR, is feasible for use among young patients with diabetes, increases adherence to recommended screening, and is cost-effective. [23][24][25][26][27][28] Our study sought to estimate the burden of DR among children, adolescents, and young adults in the US at the time of point-of-care DR screening and to identify factors associated with DR by combining data from 2 large pediatric hospitals that have robust DR screening programs. Combining data from these 2 sites allowed us to report findings from analyses that, to our knowledge, include one of the largest and most racially diverse groups of young patients with type 1 and type 2 diabetes in the US.…”

Assessment of Trained Image Grader Performance in Screening for Retinopathy Among Youth With Diabetes

“…Retinal images from 90 patients were randomly chosen from a set of 308 patients, enrolled in a previously published prospective study conducted at the Johns Hopkins Children's center. 4 Both graders independently reviewed the same retinal images from 90 patients and assigned each patient a DR grade according to the International Clinical Diabetic Retinopathy classification. 5 If the two eyes had different grades, then the eye with the higher grade was considered the DR grade for the patient.…”

“…The reference standard determination is described in our prior publication. 4 Sensitivity, specificity, and agreement were determined against this reference standard.…”