Multimodal Information Fusion for Glaucoma and Diabetic Retinopathy Classification

Li, Yihao; Daho, Mostafa El Habib; Conze, Pierre-Henri; Alhajj, Hassan; Bonnin, Sophie; Ren, Hugang; Manivannan, Niranchana; Magazzeni, Stéphanie; Tadayoni, Ramin; Cochener, Béatrice; Lamard, Mathieu

doi:10.1007/978-3-031-16525-2_6

Cited by 20 publications

(18 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our feature fusion module, as compared to schemes that perform feature fusion during feature extraction, such as the approach proposed by Li et al (2022b), our method evidently enjoys a natural advantage. This advantage arises from our approach of separately invoking pre-trained models for each branch.…”

Section: Resultsmentioning

confidence: 99%

Mstnet: method for glaucoma grading based on multimodal feature fusion of spatial relations

Wang,

Zhang

et al. 2023

Phys. Med. Biol.

View full text Add to dashboard Cite

Objective:The objective of this study is to develop an efficient multimodal learning framework for the classification of glaucoma. Glaucoma is a group of eye diseases that can result in vision loss and blindness, often due to delayed detection and treatment. Fundus images and optical coherence tomography (OCT) images have proven valuable for the diagnosis and management of glaucoma. However, current models that combine features from both modalities often lack efficient spatial relationship modeling. Approach:In this study, we propose an innovative approach to address the classification of glaucoma. We focus on leveraging the features of OCT volumes and harness the capabilities of transformer models to capture long-range spatial relationships. To achieve this, we introduce a 3D transformer model to extract features from OCT volumes, enhancing the model's effectiveness. Additionally, we employ downsampling techniques to enhance model efficiency. We then utilize the spatial feature relationships between OCT volumes and fundus images to fuse the features extracted from both sources. Main Results:Our proposed framework has yielded remarkable results, particularly in terms of glaucoma grading performance. We conducted our experiments using the GAMMA dataset, and our approach outperformed traditional feature fusion methods. By effectively modeling spatial relationships and combining OCT volume and fundus map features, our framework achieved outstanding classification results.Significance:This research is of significant importance in the field of glaucoma diagnosis and management. Efficient and accurate glaucoma classification is essential for timely intervention and prevention of vision loss. Our proposed approach, which integrates 3D transformer models, offers a novel way to extract and fuse features from OCT volumes and fundus images, ultimately enhancing the effectiveness of glaucoma classification. This work has the potential to contribute to improved patient care, particularly in the early detection and treatment of glaucoma, thereby reducing the risk of vision impairment and blindness.

show abstract

Section: Resultsmentioning

confidence: 99%

Mstnet: method for glaucoma grading based on multimodal feature fusion of spatial relations

Wang,

Zhang

et al. 2023

Phys. Med. Biol.

View full text Add to dashboard Cite

show abstract

“…A project called EviRed was presented in [39], which seeks to enhance the screening, diagnosis, and management of DR using AI. DR is a significant cause of blindness in developed nations, and the current classification based on traditional fundus photography offers limited predictive accuracy.…”

Section: Related Workmentioning

confidence: 99%

Deep learning innovations in diagnosing diabetic retinopathy: The potential of transfer learning and the DiaCNN model

Shoaib,

Emara,

Zhao

et al. 2024

Computers in Biology and Medicine

View full text Add to dashboard Cite

“…The DenseNet121 was chosen as the base CNN architecture for this study owing to its notable success in ophthalmology, particularly in classifying various ocular disorders 24,[27][28][29] . The base architecture is used for feature extraction from the input images.…”

Section: Deep Learning Architecturementioning

confidence: 99%

“…Utilizing knowledge acquired from broader dataset training, transfer learning optimizes CNN weights for targeted tasks, demonstrating effectiveness in various ophthalmic applications 22,23 . Some studies have taken this further by exploring the combination of transfer learning with data fusion to enhance the deep learning performance 16,[24][25][26] . Data fusion involves integrating information from diverse sources or modalities.…”

Section: Introductionmentioning

confidence: 99%

Deep learning for automated diagnosis of uveal melanoma

Dadzie,

Iddir,

Abtahi

et al. 2024

Ophthalmic Technologies XXXIV

View full text Add to dashboard Cite

Accurate differentiation of uveal melanoma and choroidal nevi is critical for optimal patient care, preventing unnecessary procedures for benign lesions while ensuring timely intervention for potentially malignant cases. This study aimed to validate deep learning classification of these lesions and to evaluate the impact of different color fusion options on classification performance. To evaluate the effect of color fusion options on the classification performance, we tested early fusion, intermediate fusion, and late fusion using ultra-widefield retinal images. Specificity, sensitivity, F1-score, accuracy, and the area under the curve (AUC) of a receiver operating characteristic (ROC) were used to assess the performance of the deep learning model. The results show that the color fusion options significantly impacted the deep learning classification performance, with intermediate fusion emerging as the best strategy, outperforming both singlecolor learning and the other fusion strategies. The intermediate fusion strategy had an accuracy of 89.72%, sensitivity of 85.05%, specificity of 91.64, F1 score of 0.8492 and an AUC of 0.9335. These compelling results emphasize the vast potential of deep learning to enhance the accuracy of diagnosis and classification of UM and choroidal nevi, leading to improved patient outcomes and optimized treatment strategies. By harnessing the power of deep learning and color fusion strategies, this study not only provides valuable insights into the application of these approaches in the field of ophthalmology but also highlights their critical significance in automating the classification of UM and choroidal nevi.

show abstract

Multimodal Information Fusion for Glaucoma and Diabetic Retinopathy Classification

Cited by 20 publications

References 23 publications

Mstnet: method for glaucoma grading based on multimodal feature fusion of spatial relations

Mstnet: method for glaucoma grading based on multimodal feature fusion of spatial relations

Deep learning innovations in diagnosing diabetic retinopathy: The potential of transfer learning and the DiaCNN model

Deep learning for automated diagnosis of uveal melanoma

Contact Info

Product

Resources

About