Monitoring Disease Progression With a Quantitative Severity Scale for Retinopathy of Prematurity Using Deep Learning

Taylor, Stanford; Brown, James M.; Gupta, Krishnendu; Campbell, J. Peter; Ostmo, Susan; Chan, Robison Vernon Paul; Dy, Jennifer G.; Erdoğmuş, Deni̇z; Ioannidis, Stratis; Kim, Sang J.; Kalpathy–Cramer, Jayashree; Chiang, Michael F.

doi:10.1001/jamaophthalmol.2019.2433

Cited by 97 publications

(73 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although the number of pre-plus images available for evaluation was relatively small, this may support reports that plus disease likely exists as a continuous spectrum of retinal vascular abnormality (Fig. 4),29,36 a hypothesis also suggested in recent ROP deep learning literature 29,37,38. With significant further development and validation, the algorithm may be able to assist in the objective quantification of ROP disease severity and the monitoring of disease progression.…”

Section: Discussionsupporting

confidence: 72%

Deep Learning Algorithm for Automated Diagnosis of Retinopathy of Prematurity Plus Disease

Tan

Simkin

Lai

et al. 2019

Trans. Vis. Sci. Tech.

View full text Add to dashboard Cite

Purpose: This study describes the initial development of a deep learning algorithm, ROP.AI, to automatically diagnose retinopathy of prematurity (ROP) plus disease in fundal images.Methods: ROP.AI was trained using 6974 fundal images from Australasian image databases. Each image was given a diagnosis as part of real-world routine ROP screening and classified as normal or plus disease. The algorithm was trained using 80% of the images and validated against the remaining 20% within a hold-out test set. Performance in diagnosing plus disease was evaluated against an external set of 90 images. Performance in detecting pre-plus disease was also tested. As a screening tool, the algorithm's operating point was optimized for sensitivity and negative predictive value, and its performance reevaluated.Results: For plus disease diagnosis within the 20% hold-out test set, the algorithm achieved a 96.6% sensitivity, 98.0% specificity, and 97.3% 6 0.7% accuracy. Area under the receiver operating characteristic curve was 0.993. Within the independent test set, the algorithm achieved a 93.9% sensitivity, 80.7% specificity, and 95.8% negative predictive value. For detection of pre-plus and plus disease, the algorithm achieved 81.4% sensitivity, 80.7% specificity, and 80.7% negative predictive value. Following the identification of an optimized operating point, the algorithm diagnosed plus disease with a 97.0% sensitivity and 97.8% negative predictive value.Conclusions: ROP.AI is a deep learning algorithm able to automatically diagnose ROP plus disease with high sensitivity and negative predictive value.Translational Relevance: In the context of increasing global disease burden, future development may improve access to ROP diagnosis and care.

show abstract

Section: Discussionsupporting

confidence: 72%

Deep Learning Algorithm for Automated Diagnosis of Retinopathy of Prematurity Plus Disease

Tan

Simkin

Lai

et al. 2019

Trans. Vis. Sci. Tech.

View full text Add to dashboard Cite

show abstract

“…Redd et al reported that a scale from 1 to 9 could accurately detect type 1 ROP with an area under the curve of 0.95 and, in theory, could decrease the number of ophthalmoscopic examinations by 80% in a real-world telemedicine program because most babies will have no or mild disease. 34 More recently, Taylor et al 35 implemented the i-ROP DL algorithm to assign a continuous ROP vascular severity score (1)(2)(3)(4)(5)(6)(7)(8)(9) and to classify images based on severity: no ROP, mild ROP, type 2 ROP, and preplus disease, or type 1 ROP. Using reference standard diagnoses, this retrospective cohort study concluded that the continuous ROP vascular score was associated both with the ICROP category of disease at a single point in time, and clinical progression of ROP over time.…”

Section: A Continuous Severity Score For Ropmentioning

confidence: 99%

“…Using reference standard diagnoses, this retrospective cohort study concluded that the continuous ROP vascular score was associated both with the ICROP category of disease at a single point in time, and clinical progression of ROP over time. 35 Gupta et al 36 showed that these continuous scores reflected posttreatment regression in eyes with treatment requiring-ROP. Using i-ROP data, this group also found that eyes requiring multiple treatment sessions (laser or intravitreal injection of bevacizumab) had higher pretreatment ROP vascular severity scores compared with eyes requiring only a single treatment, suggesting that treatment failure may be related to more aggressive disease or disease treated at a later stage.…”

Section: A Continuous Severity Score For Ropmentioning

confidence: 99%

“…This continuous ROP vascular score provided a more quantitative diagnosis for plus disease and was associated with both the ICROP category of disease at a single point in time and clinical progression of ROP over time. 35 The quantitative ROP vascular severity group (1–9), which was described and evaluated by Gupta et al 36 and Taylor et al 35 consistently reflects plus disease presence (i.e. more advanced ROP stages are associated with increased plus score) 2 , ROP disease progression, and posttreatment regression.…”

Section: Recommendations For Future Research In Ai Applied To Ropmentioning

confidence: 99%

See 1 more Smart Citation

Artificial Intelligence in Retinopathy of Prematurity Diagnosis

Scruggs

Chan

Kalpathy–Cramer

et al. 2020

Trans. Vis. Sci. Tech.

Self Cite

View full text Add to dashboard Cite

Citation: Scruggs BA, Chan RVP, Kalpathy-Cramer J, Chiang MF, Campbell JP. Artificial intelligence in retinopathy of prematurity diagnosis. Trans Vis Sci Tech. 2020;9(2):5, https://doi.org/10.1167/tvst.9.2.5Retinopathy of prematurity (ROP) is a leading cause of childhood blindness worldwide. The diagnosis of ROP is subclassified by zone, stage, and plus disease, with each area demonstrating significant intra-and interexpert subjectivity and disagreement. In addition to improved efficiencies for ROP screening, artificial intelligence may lead to automated, quantifiable, and objective diagnosis in ROP. This review focuses on the development of artificial intelligence for automated diagnosis of plus disease in ROP and highlights the clinical and technical challenges of both the development and implementation of artificial intelligence in the real world.

show abstract

“…The evaluation of disease severity and change over time on medical images are important and routine tasks. For example, vascular changes in retinopathy of prematurity (ROP) can be an important biomarker of disease progression and response to therapy 1,2 . For many types of pathologies, the disease process can have a wide continuous spectrum of severities, which can change over time.…”

Section: Introductionmentioning

confidence: 99%

Siamese neural networks for continuous disease severity evaluation and change detection in medical imaging

Chang

Bearce

et al. 2020

npj Digit. Med.

Self Cite

View full text Add to dashboard Cite

Using medical images to evaluate disease severity and change over time is a routine and important task in clinical decision making. Grading systems are often used, but are unreliable as domain experts disagree on disease severity category thresholds. These discrete categories also do not reflect the underlying continuous spectrum of disease severity. To address these issues, we developed a convolutional Siamese neural network approach to evaluate disease severity at single time points and change between longitudinal patient visits on a continuous spectrum. We demonstrate this in two medical imaging domains: retinopathy of prematurity (ROP) in retinal photographs and osteoarthritis in knee radiographs. Our patient cohorts consist of 4861 images from 870 patients in the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) cohort study and 10,012 images from 3021 patients in the Multicenter Osteoarthritis Study (MOST), both of which feature longitudinal imaging data. Multiple expert clinician raters ranked 100 retinal images and 100 knee radiographs from excluded test sets for severity of ROP and osteoarthritis, respectively. The Siamese neural network output for each image in comparison to a pool of normal reference images correlates with disease severity rank (ρ = 0.87 for ROP and ρ = 0.89 for osteoarthritis), both within and between the clinical grading categories. Thus, this output can represent the continuous spectrum of disease severity at any single time point. The difference in these outputs can be used to show change over time. Alternatively, paired images from the same patient at two time points can be directly compared using the Siamese neural network, resulting in an additional continuous measure of change between images. Importantly, our approach does not require manual localization of the pathology of interest and requires only a binary label for training (same versus different). The location of disease and site of change detected by the algorithm can be visualized using an occlusion sensitivity map-based approach. For a longitudinal binary change detection task, our Siamese neural networks achieve test set receiving operator characteristic area under the curves (AUCs) of up to 0.90 in evaluating ROP or knee osteoarthritis change, depending on the change detection strategy. The overall performance on this binary task is similar compared to a conventional convolutional deep-neural network trained for multi-class classification. Our results demonstrate that convolutional Siamese neural networks can be a powerful tool for evaluating the continuous spectrum of disease severity and change in medical imaging.npj Digital Medicine (2020) 3:48 ; https://doi.

show abstract

Monitoring Disease Progression With a Quantitative Severity Scale for Retinopathy of Prematurity Using Deep Learning

Cited by 97 publications

References 33 publications

Deep Learning Algorithm for Automated Diagnosis of Retinopathy of Prematurity Plus Disease

Deep Learning Algorithm for Automated Diagnosis of Retinopathy of Prematurity Plus Disease

Artificial Intelligence in Retinopathy of Prematurity Diagnosis

Siamese neural networks for continuous disease severity evaluation and change detection in medical imaging

Contact Info

Product

Resources

About