A Multimodal Classification Architecture for the Severity Diagnosis of Glaucoma Based on Deep Learning

Abstract

Glaucoma is an optic neuropathy that leads to characteristic visual field defects. However, there is no cure for glaucoma, so the diagnosis of its severity is essential for its prevention. In this paper, we propose a multimodal classification architecture based on deep learning for the severity diagnosis of glaucoma. In this architecture, a gray scale image of the visual field is first reconstructed with a higher resolution in the preprocessing stage, and more subtle feature information is provided for glaucoma diagnosis. We then use multimodal fusion technology to integrate fundus images and gray scale images of the visual field as the input of this architecture. Finally, the inherent limitation of convolutional neural networks (CNNs) is addressed by replacing the original classifier with the proposed classifier. Our architecture is trained and tested on the datasets provided by the First Affiliated Hospital of Kunming Medical University, and the results show that the proposed architecture achieves superior performance for glaucoma diagnosis.

Artificial intelligence for the detection of glaucoma with SD-OCT images: a systematic review and Meta-analysis

“…One is the single-path method, which inputs single-type data. The other is a multimodal fusion image, which is combined with two or more types of data [55] . A number of studies have shown that multimodal imaging based on DL can detect…”

“…3, Mar. 18, 2024 www.ijo.cn Tel: 8629-82245172 8629-82210956 Email: ijopress@163.com glaucoma with higher accuracy, which can further improve the performance of glaucoma diagnosis [55][56][57] . OCT is a non-invasive imaging technique [58] .…”

Multimodal and multi-omics-based deep learning model for screening of optic neuropathy

“…Their study demonstrated that a multi-modal system was significantly superior to single-mode systems. Kihara's DL system, which integrated OCT and visual field images, also showed excellent diagnostic performance, with an AUC of 0.90–0.95 [ 21 ]; [ 22 ]. The development of multimodal algorithms is a developing trend.…”

SMMF: a self-attention-based multi-parametric MRI feature fusion framework for the diagnosis of bladder cancer grading

“…Figure 1A demonstrates the input-level fusion strategy, which integrates various modal images into a single dataset, allowing the neural network to utilize all information from each image and preserve the original features to the fullest extent. The potential of input-level fusion has been shown in several studies to obtain comprehensive features ( 35 , 36 ). As shown in Figure 1B , feature-level fusion is a simpler feature fusion method, which usually uses the “Concate” or “Add” method to stitch together the feature maps extracted from different neural network branches.…”