DT-Net: Joint Dual-Input Transformer and CNN for Retinal Vessel Segmentation

Abstract

Retinal vessel segmentation in fundus images plays an essential role in the screening, diagnosis, and treatment of many diseases. The acquired fundus images generally have the following problems: uneven illumination, high noise, and complex structure. It makes vessel segmentation very challenging. Previous methods of retinal vascular segmentation mainly use convolutional neural networks on U Network (U-Net) models, and they have many limitations and shortcomings, such as the loss of microvascular details at the end of the vessels. We address the limitations of convolution by introducing the transformer into retinal vessel segmentation. Therefore, we propose a hybrid method for retinal vessel segmentation based on modulated deformable convolution and the transformer, named DT-Net. Firstly, multi-scale image features are extracted by deformable convolution and multi-head selfattention (MHSA). Secondly, image information is recovered, and vessel morphology is refined by the proposed transformer decoder block. Finally, the local prediction results are obtained by the side output layer. The accuracy of the vessel segmentation is improved by the hybrid loss function. Experimental results show that our method obtains good segmentation performance on Specificity (SP), Sensitivity (SE), Accuracy (ACC), Curve (AUC), and F1-score on three publicly available fundus datasets such as DRIVE, STARE, and CHASE_DB1.

TD Swin-UNet: Texture-Driven Swin-UNet with Enhanced Boundary-Wise Perception for Retinal Vessel Segmentation

“…They designed the LGFA architecture to capture long-range feature dependencies and proposed bidirectional weighted feature fusion (BWF) with adaptive lateral feature fusion (ALFF) for features of varying scales. Jia et al [26] proposed DT-Net, which merges deformable convolution with the transformer and incorporates MSA in the decoder to capture long-range dependencies and important features within blood vessels. In addition to fusing the transformer with the CNN, some scholars have also focused on enhancing transformer structures or attention mechanisms to improve segmentation accuracy.…”