Reproduction study using public data of: Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs

Voets, Mike; Møllersen, Kajsa; Bongo, Lars Ailo

doi:10.1371/journal.pone.0217541

Cited by 153 publications

(133 citation statements)

References 25 publications

Supporting

Mentioning

121

Contrasting

Unclassified

Order By: Relevance

“…It can detect subclinical and discrete features appearing below the threshold of a human observer and quantify minimal differences in feature expression. Recently, a CNN was trained on fundus images to screen DR [37][38][39][40][41][42] or age-related macular degeneration [43]. Moreover, a more sophisticated CNN (Google Inceptionv3) has been trained on datasets from the UK Biobank [44] and EyePACS [45] cohorts to detect cardiovascular risk factors from retinal images such as age, gender, hypertension, and smoking status [46].…”

Section: Introductionmentioning

confidence: 99%

Exploring the effect of hypertension on retinal microvasculature using deep learning on East Asian population

Dai

et al. 2020

PLoS ONE

View full text Add to dashboard Cite

Hypertension is the leading risk factor of cardiovascular disease and has profound effects on both the structure and function of the microvasculature. Abnormalities of the retinal vasculature may reflect the degree of microvascular damage due to hypertension, and these changes can be detected with fundus photographs. This study aimed to use deep learning technique that can detect subclinical features appearing below the threshold of a human observer to explore the effect of hypertension on morphological features of retinal microvasculature. We collected 2012 retinal photographs which included 1007 from patients with a diagnosis of hypertension and 1005 from normotensive control. By method of vessel segmentation, we removed interference information other than retinal vasculature and contained only morphological information about blood vessels. Using these segmented images, we trained a small convolutional neural networks (CNN) classification model and used a deep learning technique called Gradient-weighted Class Activation Mapping (Grad-CAM) to generate heat maps for the class "hypertension". Our model achieved an accuracy of 60.94%, a specificity of 51.54%, a precision of 59.27%, and a recall of 70.48%. The AUC was 0.6506. In the heat maps for the class "hypertension", red patchy areas were mainly distributed on or around arterial/venous bifurcations. This indicated that the model has identified these regions as being the most important for predicting hypertension. Our study suggested that the effect of hypertension on retinal microvascular morphology mainly occurred at branching of vessels. The change of the branching pattern of retinal vessels was probably the most significant in response to elevated blood pressure.

show abstract

Section: Introductionmentioning

confidence: 99%

Exploring the effect of hypertension on retinal microvasculature using deep learning on East Asian population

Dai

et al. 2020

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…Here we achieved specificity and sensitivity as high as 89% and 98%, using a bi-classification grading scheme. Here we have shown [Table 7 & 8] that by tweaking the post processing of the outcome of a CNN, we have outperformed the previously published best performance of Kaggl EyePACS, which was later failed to be replicated(35).…”

Section: Resultsmentioning

confidence: 85%

Towards implementation of AI in New Zealand national screening program: Cloud-based, Robust, and Bespoke

Xie

Yang

Squirrell

et al. 2019

Preprint

View full text Add to dashboard Cite

25Convolutional Neural Networks (CNN)s have become a prominent method of AI 26 implementation in medical classification tasks. Grading Diabetic Retinopathy (DR) has been 27 at the forefront of the development of AI for ophthalmology. However, major obstacles remain 28 in the generalization of these CNN's onto real-world DR screening programs. We believe these 29 difficulties are due to use of 1) small training datasets (<5,000 images), 2) private and 'curated' 30 repositories, 3) offline CNN implementation methods, while 4) relying on accuracy measured 31 as area under the curve (AUC) as the sole measure of CNN performance. 32 To address these issues, the public EyePACS Kaggle Diabetic Retinopathy dataset was 33 uploaded onto Microsoft Azure™ cloud platform. Two CNNs were trained as a "Quality 34 Assurance", and a "Classifier". The "Classifier" CNN performance was then tested both on 35 'un-curated' as well as the 'curated' test set created by the "Quality Assessment" CNN. Finally, 36 the sensitivity of the "Classifier" CNNs was boosted post-training using two post-training 37 techniques. 38Our "Classifier" CNN proved to be robust, as its performance was similar on 'curated' and 'un-39 curated' sets. The implementation of 'cascading thresholds' and 'max margin' techniques led 40 to significant improvements in the "Classifier" CNN's sensitivity, while also enhancing the 41 specificity of other grades. 42 43 4 44 45It is estimated that by 2040, nearly 600 million people will have diabetes worldwide(1). 46Diabetic retinopathy (DR) is a common diabetes-related microvascular complication, and is 47 the leading cause of preventable blindness in people of working age worldwide(2, 3). It has 48 been estimated that the overall prevalence of non-vision-threatening DR, vision-threatening 49 DR and the blinding diabetic eye disease were 34·6%, 10·2%, and 6·8% respectively (3-6). 50Clinical trials have shown that the risk of DR progression can be significantly reduced by 51 controlling major risk factors such as hyperglycaemia and hypertension (7-9). It is further 52 estimated that screening, appropriate referral and treatment can reduce the vision loss from DR 53 by 50% (10-12). However, DR screening programs are expensive to set up and administrate. It 54 is estimated that even in developed countries, these programs do not reach up to 30% of the 55 diabetic population (13, 14). 56 Artificial intelligence (AI) and its subcategory of deep learning have gained popularity in 57 medical screening programs, including DR screening. In deep learning, a convolutional neural 58 network (CNN) is designed and trained based on large datasets of ground truth data and labels. 59The CNN algorithm adjusts its weights and discovers which features to extract from medical 60 data (e.g. fundus photos) to achieve the best classification accuracy, when compared to human 61 performance (15-20). CNNs use layers with convolutions, which are defined as mathematical 62 functions that use filters to extract features from an image (21-23). The out...

show abstract

“…Here we have shown [ Tables 7 & 8] that by adjusting the post processing of the outcome of a CNN, we have outperformed the previously published best performance of Kaggle EyePACS, which was later failed to be replicated [37].…”

Section: Sensitivity Upliftmentioning

confidence: 84%

Towards implementation of AI in New Zealand national diabetic screening program: Cloud-based, robust, and bespoke

et al. 2020

View full text Add to dashboard Cite

Convolutional Neural Networks (CNNs) have become a prominent method of AI implementation in medical classification tasks. Grading Diabetic Retinopathy (DR) has been at the forefront of the development of AI for ophthalmology. However, major obstacles remain in the generalization of these CNNs onto real-world DR screening programs. We believe these difficulties are due to use of 1) small training datasets (<5,000 images), 2) private and 'curated' repositories, 3) locally implemented CNN implementation methods, while 4) relying on measured Area Under the Curve (AUC) as the sole measure of CNN performance. To address these issues, the public EyePACS Kaggle Diabetic Retinopathy dataset was uploaded onto Microsoft Azure™ cloud platform. Two CNNs were trained; 1 a "Quality Assurance", and 2. a "Classifier". The Diabetic Retinopathy classifier CNN (DRCNN) performance was then tested both on 'un-curated' as well as the 'curated' test set created by the "Quality Assessment" CNN model. Finally, the sensitivity of the DRCNNs was boosted using two post-training techniques. Our DRCNN proved to be robust, as its performance was similar on 'curated' and 'un-curated' test sets. The implementation of 'cascading thresholds' and 'max margin' techniques led to significant improvements in the DRCNN's sensitivity, while also enhancing the specificity of other grades. OPEN ACCESSCitation: Xie L, Yang S, Squirrell D, Vaghefi E (2020) Towards implementation of AI in New Zealand national diabetic screening program: Cloud-based, robust, and bespoke. PLoS ONE 15 (4): e0225015. https://doi.org/10.

show abstract

Reproduction study using public data of: Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs

Cited by 153 publications

References 25 publications

Exploring the effect of hypertension on retinal microvasculature using deep learning on East Asian population

Exploring the effect of hypertension on retinal microvasculature using deep learning on East Asian population

Towards implementation of AI in New Zealand national screening program: Cloud-based, Robust, and Bespoke

Towards implementation of AI in New Zealand national diabetic screening program: Cloud-based, robust, and bespoke

Contact Info

Product

Resources

About