Vison transformer adapter-based hyperbolic embeddings for multi-lesion segmentation in diabetic retinopathy

Abstract

Diabetic Retinopathy (DR) is a major cause of blindness worldwide. Early detection and treatment are crucial to prevent vision loss, making accurate and timely diagnosis critical. Deep learning technology has shown promise in the automated diagnosis of DR, and in particular, multi-lesion segmentation tasks. In this paper, we propose a novel Transformer-based model for DR segmentation that incorporates hyperbolic embeddings and a spatial prior module. The proposed model is primarily built on a traditional Vision Transformer encoder and further enhanced by incorporating a spatial prior module for image convolution and feature continuity, followed by feature interaction processing using the spatial feature injector and extractor. Hyperbolic embeddings are used to classify feature matrices from the model at the pixel level. We evaluated the proposed model’s performance on the publicly available datasets and compared it with other widely used DR segmentation models. The results show that our model outperforms these widely used DR segmentation models. The incorporation of hyperbolic embeddings and a spatial prior module into the Vision Transformer-based model significantly improves the accuracy of DR segmentation. The hyperbolic embeddings enable us to better capture the underlying geometric structure of the feature matrices, which is important for accurate segmentation. The spatial prior module improves the continuity of the features and helps to better distinguish between lesions and normal tissues. Overall, our proposed model has potential for clinical use in automated DR diagnosis, improving accuracy and speed of diagnosis. Our study shows that the integration of hyperbolic embeddings and a spatial prior module with a Vision Transformer-based model improves the performance of DR segmentation models. Future research can explore the application of our model to other medical imaging tasks, as well as further optimization and validation in real-world clinical settings.

Diabetic Retinopathy Lesion Segmentation Method Based on Multi-Scale Attention and Lesion Perception

“…In this paper, MLNet is compared with seven state-of-the-art DR lesion segmentation methods, including M2MRF [22], L-Seg [37], RTNet [9], HEDNet [38], Swin Transformer [39], VTA [20], and Guo et al [11]. Table 1 presents the evaluation metrics and mean evaluation metrics' results of MLNet and the seven advanced methods on the DDR dataset.…”

“…Furthermore, the introduction of innovative approaches like RTNet by Huang et al [9], hyperbolic space-based Transformer by Wang et al [20], SSMD-Unet by Ullah et al [21], and M2MRF operator by Liu et al [22] has contributed to further advancements in DR lesion segmentation. These methods have leveraged self-attention, cross-attention, relation Transformer blocks, hyperbolic embeddings, and auxiliary reconstruction tasks to enhance segmentation accuracy and address specific challenges associated with scale discrepancy and optimal feature representation.…”

UC-stack: a deep learning computer automatic detection system for diabetic retinopathy classification

“…CNN and SVM are used to classify DR Lesions. Wang et al (2023) proposed the introduction of hyperbolic embeddings and spatial prior modules into the Vision Transformer model, significantly improving the accuracy of DR segmentation.…”