Automated Analysis of Retinal Images for Detection of Referable Diabetic Retinopathy

Abràmoff, Michael D.; Folk, James C.; Han, Dennis P.; Walker, Jonathan D.; Williams, David F.; Russell, Stephen R.; Massin, Pascale; Cochener, Béatrice; Gain, Philippe; Tang, Li; Lamard, Mathieu; Moga, Daniela C.; Quellec, Gwénolé; Niemeijer, Meindert

doi:10.1001/jamaophthalmol.2013.1743

Cited by 364 publications

(272 citation statements)

References 30 publications

Supporting

Mentioning

251

Contrasting

Unclassified

Order By: Relevance

“…16,36 Most recently, the IDx-DR was validated on 1748 eyes with single-field 45°colour fundus images acquired in French primary care clinics. 36 In this study, the proportion of referable diabetic retinopathy was high, at 21.7%, and sensitivity and specificity was reported as 96.8% and 59.4%, respectively.…”

Section: Idx-drmentioning

confidence: 99%

“…Three software systems were selected from a literature search and discussion with experts and all three were agreed to participate in the study: iGradingM, 31 Retmarker and IDx-DR. 36 For commercial reasons, IDx, LLC withdrew from the study before the analysis, and in 2013 another software system (EyeArt) asked to join the study, confirming that it METHODS NIHR Journals Library www.journalslibrary.nihr.ac.uk would meet CE-mark eligibility criteria. Three systems (iGradingM, Retmarker and EyeArt) then processed all images from all screening episodes.…”

Section: Automated Grading Systemsmentioning

confidence: 99%

See 1 more Smart Citation

An observational study to assess if automated diabetic retinopathy image assessment software can replace one or more steps of manual imaging grading and to determine their cost-effectiveness

Tufail

Kapetanakis

Salas-Vega

et al. 2016

Health Technol Assess

106

128

View full text Add to dashboard Cite

Background Diabetic retinopathy screening in England involves labour-intensive manual grading of retinal images. Automated retinal image analysis systems (ARIASs) may offer an alternative to manual grading. Objectives To determine the screening performance and cost-effectiveness of ARIASs to replace level 1 human graders or pre-screen with ARIASs in the NHS diabetic eye screening programme (DESP). To examine technical issues associated with implementation. Design Observational retrospective measurement comparison study with a real-time evaluation of technical issues and a decision-analytic model to evaluate cost-effectiveness. Setting A NHS DESP. Participants Consecutive diabetic patients who attended a routine annual NHS DESP visit. Interventions Retinal images were manually graded and processed by three ARIASs: iGradingM (version 1.1; originally Medalytix Group Ltd, Manchester, UK, but purchased by Digital Healthcare, Cambridge, UK, at the initiation of the study, purchased in turn by EMIS Health, Leeds, UK, after conclusion of the study), Retmarker (version 0.8.2, Retmarker Ltd, Coimbra, Portugal) and EyeArt (Eyenuk Inc., Woodland Hills, CA, USA). The final manual grade was used as the reference standard. Arbitration on a subset of discrepancies between manual grading and the use of an ARIAS by a reading centre masked to all grading was used to create a reference standard manual grade modified by arbitration. Main outcome measures Screening performance (sensitivity, specificity, false-positive rate and likelihood ratios) and diagnostic accuracy [95% confidence intervals (CIs)] of ARIASs. A secondary analysis explored the influence of camera type and patients’ ethnicity, age and sex on screening performance. Economic analysis estimated the cost per appropriate screening outcome identified. Results A total of 20,258 patients with 102,856 images were entered into the study. The sensitivity point estimates of the ARIASs were as follows: EyeArt 94.7% (95% CI 94.2% to 95.2%) for any retinopathy, 93.8% (95% CI 92.9% to 94.6%) for referable retinopathy and 99.6% (95% CI 97.0% to 99.9%) for proliferative retinopathy; and Retmarker 73.0% (95% CI 72.0% to 74.0%) for any retinopathy, 85.0% (95% CI 83.6% to 86.2%) for referable retinopathy and 97.9% (95% CI 94.9 to 99.1%) for proliferative retinopathy. iGradingM classified all images as either ‘disease’ or ‘ungradable’, limiting further iGradingM analysis. The sensitivity and false-positive rates for EyeArt were not affected by ethnicity, sex or camera type but sensitivity declined marginally with increasing patient age. The screening performance of Retmarker appeared to vary with patient’s age, ethnicity and camera type. Both EyeArt and Retmarker were cost saving relative to manual grading either as a replacement for level 1 human grading or used prior to level 1 human grading, although the latter was less cost-effective. A threshold analysis testing the highest ARIAS cost per patient before which ARIASs became more expensive per appropriate outcome than human grading, when used to replace level 1 grader, was Retmarker £3.82 and EyeArt £2.71 per patient. Limitations The non-randomised study design limited the health economic analysis but the same retinal images were processed by all ARIASs in this measurement comparison study. Conclusions Retmarker and EyeArt achieved acceptable sensitivity for referable retinopathy and false-positive rates (compared with human graders as reference standard) and appear to be cost-effective alternatives to a purely manual grading approach. Future work is required to develop technical specifications to optimise deployment and address potential governance issues. Funding The National Institute for Health Research (NIHR) Health Technology Assessment programme, a Fight for Sight Grant (Hirsch grant award) and the Department of Health’s NIHR Biomedical Research Centre for Ophthalmology at Moorfields Eye Hospital and the University College London Institute of Ophthalmology.

show abstract

Section: Idx-drmentioning

confidence: 99%

Section: Automated Grading Systemsmentioning

confidence: 99%

An observational study to assess if automated diabetic retinopathy image assessment software can replace one or more steps of manual imaging grading and to determine their cost-effectiveness

Tufail

Kapetanakis

Salas-Vega

et al. 2016

Health Technol Assess

106

128

View full text Add to dashboard Cite

show abstract

“…This research group is responsible of important scientific publications in this field by using both classical (Abràmoff et al, 2013(Abràmoff et al, , 2010Niemeijer et al, 2010) and novel approaches (Abràmoff et al, 2016;Costa et al, 2017).…”

Section: International Groupsmentioning

confidence: 99%

“…In this part of the work, the green channel of a RGB image was chosen to perform the vessel inpainting because it is the component commonly used to detect the lesions (the objective of a computer-aided diagnosis system) (Abràmoff et al, 2013;Walter et al, 2002;Zhang et al, 2014).…”

Section: Grey-level Experimentsmentioning

confidence: 99%

See 1 more Smart Citation

Fundus image analysis for automatic screening of ophthalmic pathologies.

Granero¹

View full text Add to dashboard Cite

AcknowledgmentsEste documento es el resultado de cuatro años de esfuerzo y dedicación que han propiciado mi evolución tanto a nivel profesional como personal. El principal motivo de dicho desarrollo han sido todas esas personas que han formado parte de este camino o que de alguna forma se han cruzado en el mismo. A todos ellos quiero dedicarles unas palabras de agradecimiento.En primer lugar, quiero agradecer a Valery, ella es la responsable de que yo esté ahora mismo escribiendo estas líneas. Me brindó la oportunidad de mi vida hace ya más de cinco años y me ha ido formando tanto en la vertiente investigadora como docente de la misma forma que una madre educa e inculca los valores esenciales a su hijo. Gracias por todo Valery, juntos cerramos una etapa clave en mi formación y juntos abrimos una nueva cargada de ilusión. Espero poder estar siempre a tu lado.También quiero agradecer todo el apoyo, energía y momentos divertidos que me han brindado mis compañeros de laboratorio. Gracias Sandra por todo lo que me has enseñado desde el primer momento en el que llegué hasta el día de hoy, no sé qué haría yo sin ti en esa sala. Gracias Fer, Félix, Andrés, Reinier, Ángel, Gabri, Fran P., Rober, Jorge y Adri por todos los consejos aportados y por amenizar como sólo vosotros sabéis mi día a día. También quiero recordar y agradecer a los que decidieron emprender otro rumbo: Javi, Conchi, Pablo, Eliseo, Fran M. y Miguel.De igual forma quiero mostrar mi gratitud hacia varias personas que han colaborado de una u otra forma en este trabajo. A Jesús Angulo, Débora Gil y Ramón Baldrich por su supervisión del trabajo desarrollado en las estancias llevadas a cabo durante esta etapa. A Kjersti Engan por aportar sus conocimientos sobre Dictionary Learning & Sparse Representation que han culminado en diversas publicaciones. A Rafael Molina por permitirme adentrarme en el fascinante mundo de los Procesos Gaussianos y sus múltiples aplicaciones.Gracias a mi madre Jacqueline, mi hermano Fran, mis abuelos Mª Carmen y Paco, mi padre Luis y Eva que siempre me han brindado su apoyo incondicional. Como bien sabéis sois fundamentales para mí, por ello ésta tesis también va dedicada a vosotros. No me puedo olvidar de Rosa que ocupa una parte grande de mi corazón. Os quiero.A Duke y Tom por todas esas horas de compañía desinteresada que me han ofrecido durante sus vidas y en especial en la época de redacción de este manuscrito.Por último, quiero dedicar especialmente este trabajo al motor de mi vida, a la persona que tiene que lidiar conmigo en los momentos más difíciles, a la persona que siempre me acaba sacando una sonrisa, a la persona que me enseña día tras día lo que es amor, fuerza, lucha, perseverancia e infinidad de cualidades que la definen. Gracias por todo cariño, sin tu apoyo estoy seguro de que esto no hubiera sido posible. Durante estos 10 años hemos abierto y cerrado muchas etapas juntos pero lo más ilusionante son todas aquellas que nos quedan por vivir. vi AbstractIn last years, the number of blindness cases has been significan...

show abstract

Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs

Gulshan

Peng

Coram

et al. 2016

JAMA

5,720

3,664

View full text Add to dashboard Cite

IMPORTANCEDeep learning is a family of computational methods that allow an algorithm to program itself by learning from a large set of examples that demonstrate the desired behavior, removing the need to specify rules explicitly. Application of these methods to medical imaging requires further assessment and validation.OBJECTIVE To apply deep learning to create an algorithm for automated detection of diabetic retinopathy and diabetic macular edema in retinal fundus photographs. DESIGN AND SETTINGA specific type of neural network optimized for image classification called a deep convolutional neural network was trained using a retrospective development data set of 128 175 retinal images, which were graded 3 to 7 times for diabetic retinopathy, diabetic macular edema, and image gradability by a panel of 54 US licensed ophthalmologists and ophthalmology senior residents between May and December 2015. The resultant algorithm was validated in January and February 2016 using 2 separate data sets, both graded by at least 7 US board-certified ophthalmologists with high intragrader consistency.EXPOSURE Deep learning-trained algorithm. MAIN OUTCOMES AND MEASURESThe sensitivity and specificity of the algorithm for detecting referable diabetic retinopathy (RDR), defined as moderate and worse diabetic retinopathy, referable diabetic macular edema, or both, were generated based on the reference standard of the majority decision of the ophthalmologist panel. The algorithm was evaluated at 2 operating points selected from the development set, one selected for high specificity and another for high sensitivity. RESULTSThe EyePACS-1 data set consisted of 9963 images from 4997 patients (mean age, 54.4 years; 62.2% women; prevalence of RDR, 683/8878 fully gradable images [7.8%]); the Messidor-2 data set had 1748 images from 874 patients (mean age, 57.6 years; 42.6% women; prevalence of RDR, 254/1745 fully gradable images [14.6%]). For detecting RDR, the algorithm had an area under the receiver operating curve of 0.991 (95% CI, 0.988-0.993) for EyePACS-1 and 0.990 (95% CI, 0.986-0.995) for Messidor-2. Using the first operating cut point with high specificity, for EyePACS-1, the sensitivity was 90.3% (95% CI, 87.5%-92.7%) and the specificity was 98.1% (95% CI, 97.8%-98.5%). For Messidor-2, the sensitivity was 87.0% (95% CI, 81.1%-91.0%) and the specificity was 98.5% (95% CI, 97.7%-99.1%). Using a second operating point with high sensitivity in the development set, for EyePACS-1 the sensitivity was 97.5% and specificity was 93.4% and for Messidor-2 the sensitivity was 96.1% and specificity was 93.9%. CONCLUSIONS AND RELEVANCEIn this evaluation of retinal fundus photographs from adults with diabetes, an algorithm based on deep machine learning had high sensitivity and specificity for detecting referable diabetic retinopathy. Further research is necessary to determine the feasibility of applying this algorithm in the clinical setting and to determine whether use of the algorithm could lead to improved care and outcomes compared with curren...

show abstract

Automated Analysis of Retinal Images for Detection of Referable Diabetic Retinopathy

Cited by 364 publications

References 30 publications

An observational study to assess if automated diabetic retinopathy image assessment software can replace one or more steps of manual imaging grading and to determine their cost-effectiveness

An observational study to assess if automated diabetic retinopathy image assessment software can replace one or more steps of manual imaging grading and to determine their cost-effectiveness

Fundus image analysis for automatic screening of ophthalmic pathologies.

Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs

Contact Info

Product

Resources

About