Diabetic Macular Edema Detection Using End-to-End Deep Fusion Model and Anatomical Landmark Visualization on an Edge Computing Device

Abstract

PurposeDiabetic macular edema (DME) is a common cause of vision impairment and blindness in patients with diabetes. However, vision loss can be prevented by regular eye examinations during primary care. This study aimed to design an artificial intelligence (AI) system to facilitate ophthalmology referrals by physicians.MethodsWe developed an end-to-end deep fusion model for DME classification and hard exudate (HE) detection. Based on the architecture of fusion model, we also applied a dual model which included an independent classifier and object detector to perform these two tasks separately. We used 35,001 annotated fundus images from three hospitals between 2007 and 2018 in Taiwan to create a private dataset. The Private dataset, Messidor-1 and Messidor-2 were used to assess the performance of the fusion model for DME classification and HE detection. A second object detector was trained to identify anatomical landmarks (optic disc and macula). We integrated the fusion model and the anatomical landmark detector, and evaluated their performance on an edge device, a device with limited compute resources.ResultsFor DME classification of our private testing dataset, Messidor-1 and Messidor-2, the area under the receiver operating characteristic curve (AUC) for the fusion model had values of 98.1, 95.2, and 95.8%, the sensitivities were 96.4, 88.7, and 87.4%, the specificities were 90.1, 90.2, and 90.2%, and the accuracies were 90.8, 90.0, and 89.9%, respectively. In addition, the AUC was not significantly different for the fusion and dual models for the three datasets (p = 0.743, 0.942, and 0.114, respectively). For HE detection, the fusion model achieved a sensitivity of 79.5%, a specificity of 87.7%, and an accuracy of 86.3% using our private testing dataset. The sensitivity of the fusion model was higher than that of the dual model (p = 0.048). For optic disc and macula detection, the second object detector achieved accuracies of 98.4% (optic disc) and 99.3% (macula). The fusion model and the anatomical landmark detector can be deployed on a portable edge device.ConclusionThis portable AI system exhibited excellent performance for the classification of DME, and the visualization of HE and anatomical locations. It facilitates interpretability and can serve as a clinical reference for physicians. Clinically, this system could be applied to diabetic eye screening to improve the interpretation of fundus imaging in patients with DME.

The Present and Future of Artificial Intelligence-Based Medical Image in Diabetes Mellitus: Focus on Analytical Methods and Limitations of Clinical Use

“…The deep fusion model was evaluated using an edge device with limited computer resources that could be mounted on a portable device and used easily by the patient for an eye examination at an appropriate time. 27 Besides, that study reported a good performance in classifying DME through the deep fusion model, with the model reporting a sensitivity of 96% and a specificity of 90%. 27 …”

“…As the number of studies related to medical images based on deep learning, including a convolution neural network (CNN), has increased significantly, many CNN-based medical imaging studies using fundoscopy or optical coherence tomography (OCT) have been reported since 2019 ( Table 1 ). 18 19 20 21 22 23 24 25 26 27 28 29 30 This was followed by diabetic foot (7.9%, 18/227 cases) and diabetic neuropathy (2.7%, 6/227 cases) ( Fig. 3A ).…”

Innovations in Diabetic Macular Edema Management: A Comprehensive Review of Automated Quantification and Anti-vascular Endothelial Growth Factor Intervention

“…AI algorithms designed for the automated detection of exudates and macula in DME contribute to streamlining the diagnostic process. Furthermore, this automated detection aids in grading the severity of the condition, providing valuable insights for clinicians managing DME cases [ 26 ]. Predicting treatment responses in patients with DME, particularly in the context of anti-VEGF therapy, has been a focus of exploration for AI techniques.…”

“…The proposal of an end-to-end deep fusion model represents a cutting-edge approach to DME detection. Leveraging anatomical landmarks and operating on edge computing devices, this model aims to enhance the accuracy and efficiency of DME detection [ 26 ]. This holistic approach combines various deep-learning techniques for a more comprehensive assessment (Table 1 ).…”

“…Additionally, AI has been used to quantify fluid resolution and assess visual acuity gain in DME patients, providing valuable insights for treatment evaluation [ 27 ]. Furthermore, end-to-end deep fusion models have shown promise in detecting DME, with excellent performance in classifying the condition and visualizing anatomical landmarks [ 26 ]. These advancements demonstrate the potential of AI and ML in enhancing the efficiency and effectiveness of DME management, ultimately contributing to improved patient care and outcomes.…”