Spider U-Net: Incorporating Inter-Slice Connectivity Using LSTM for 3D Blood Vessel Segmentation

Abstract

Blood vessel segmentation (BVS) of 3D medical imaging such as computed tomography and magnetic resonance angiography (MRA) is an essential task in the clinical field. Automation of 3D BVS using deep supervised learning is being researched, and U-Net-based approaches, which are considered as standard for medical image segmentation, are proposed a lot. However, the inherent characteristics of blood vessels, e.g., they are complex and narrow, as well as the resolution and sensitivity of the imaging modalities increases the difficulty of 3D BVS. We propose a novel U-Net-based model named Spider U-Net for 3D BVS that considers the connectivity of the blood vessels between the axial slices. To achieve this, long short-term memory (LSTM), which can capture the context of the consecutive data, is inserted into the baseline model. We also propose a data feeding strategy that augments data and makes Spider U-Net stable. Spider U-Net outperformed 2D U-Net, 3D U-Net, and the fully convolutional network-recurrent neural network (FCN-RNN) in dice coefficient score (DSC) by 0.048, 0.077, and 0.041, respectively, for our in-house brain MRA dataset and also achieved the highest DSC for two public datasets. The results imply that considering inter-slice connectivity with LSTM improves model performance in the 3D BVS task.

Beijing Language and Culture University, China

“…As a convolutional neural network, derivatives of U-Net have been widely applied to various vascular segmentation tasks. For example, 2D [12]- [14] or 3D [15], [16] methods with modifications made mainly to the U-Net backbone network to achieve better segmentation results. They use the structure of an encoder-decoder and introduce popular structural components such as attention mechanism [12], [15], [17], atrous convolution [13], [18], gate structure [16], [17].…”

“…For example, 2D [12]- [14] or 3D [15], [16] methods with modifications made mainly to the U-Net backbone network to achieve better segmentation results. They use the structure of an encoder-decoder and introduce popular structural components such as attention mechanism [12], [15], [17], atrous convolution [13], [18], gate structure [16], [17]. Liu et al information in coronary artery DSA segmentation.…”

Deep-tomography: iterative velocity model building with deep learning

“…We investigated, for the purposed problem, the effects of changing the convolutional kernel's sizes, which partially solves the long-range dependency between velocity errors and the correspondent events observed in the migrated data. A possible alternative to accounting in the network architecture for long-range dependence between structures in output and input could be the use of recurrent models that has the potential of keeping information from the velocity errors presented in the layers above a determined depth (Chen et al, 2016;Lee et al, 2021). One important aspect is that…”

Novel Application of Long Short-Term Memory Network for 3D to 2D Retinal Vessel Segmentation in Adaptive Optics—Optical Coherence Tomography Volumes

“…In comparison to the added benefit demonstrated by the 3D-2D architecture of Li et al over baseline UNet, we achieve similar improvement of 0.04 with a Dice coefficient of 0.69 with our LSTM-UNet relative to 0.65 in the UNet alone. Lee et al developed a Spider U-Net, which similarly uses bidirectional convolutional LSTM to capture inter-slice connectivity, but employs the LSTM in between the encoding and decoding path of several UNet modules for each slice [46]. This architecture was trained and evaluated on multiple modalities, specifically Brain MRA, Abdomen CT and Cardiac MRI, for 3D-3D segmentation of blood vessels with Dice coefficients for Spider U-Net improving over 2D UNet by 0.05, 0.13, and 0.06, respectively for each dataset.…”