TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation

Chen, Jieneng; Lu, Yongyi; Yu, Qihang; Luo, Xiangde; Adeli, Ehsan; Wang, Yan; Lü, Le; Yuille, Alan; Zhou, Yuyin

doi:10.48550/arxiv.2102.04306

Cited by 1,034 publications

(1,389 citation statements)

References 20 publications

Supporting

Mentioning

1,380

Contrasting

Unclassified

Order By: Relevance

“…The most recently proposed TransUNet [49] for abdominal organ segmentation achieved a comparable overall average Dice value of 81.9%, but with significant performance drops comparing to the proposed network in ASSD (3.03 mm vs. 1.03 mm) and 95HD (13.25 mm vs. 5.40 mm). The Tran-sUNet employs 2D UNet as backbone and incorporates vision transformers (ViT) [52] into the encoder.…”

Section: A Comparison With Existing Modelsmentioning

confidence: 94%

“…We compared the segmentation performance of the proposed network to those of seven deep-learning-based models, including two baseline models [11], [12], two well-established models for medical image segmentation [13], [14], and three state-of-the-art abdominal organ segmentation models [25], [49], [20].…”

Section: Comparison With Existing Modelsmentioning

confidence: 99%

“…• TransUNet [49]: is a recent work that combines Transformer [51] and UNet [10] for 2D medical image segmentation. Specifically, the TransUNet introduces global • DenseVNet [20]: is based on the VNet architecture but additionally incorporates batch-wise spatial dropout, dense feature stacks, dilated convolutions and a spatial prior for abdominal organ segmentation 6 .…”

Section: Comparison With Existing Modelsmentioning

confidence: 99%

“…The average Dice scores across all organs of the UNet and VNet 5 We implemented the TransUnet using publicly available repository (https://github.com/Beckschen/TransUNet). Following the default setting in TransUNet [49], we used the patch size of 16 × 16 pixels. We set the input resolution as 160 × 160 pixels.…”

Section: A Comparison With Existing Modelsmentioning

confidence: 99%

See 3 more Smart Citations

Recurrent Feature Propagation and Edge Skip-Connections for Automatic Abdominal Organ Segmentation

Zhan¹,

Dong²,

Wang³

2022

Preprint

View full text Add to dashboard Cite

Automatic segmentation of abdominal organs in computed tomography (CT) images can support radiation therapy and image-guided surgery workflows. Developing of such automatic solutions remains challenging mainly owing to complex organ interactions and blurry boundaries in CT images.To address these issues, we focus on effective spatial context modeling and explicit edge segmentation priors. Accordingly, we propose a 3D network with four main components trained end-to-end including shared encoder, edge detector, decoder with edge skip-connections (ESCs) and recurrent feature propagation head (RFP-Head). To capture wide-range spatial dependencies, the RFP-Head propagates and harvests local features through directed acyclic graphs (DAGs) formulated with recurrent connections in an efficient slice-wise manner, with regard to spatial arrangement of image units. To leverage edge information, the edge detector learns edge prior knowledge specifically tuned for semantic segmentation by exploiting intermediate features from the encoder with the edge supervision. The ESCs then aggregate the edge knowledge with multi-level decoder features to learn a hierarchy of discriminative features explicitly modeling complementarity between organs' interiors and edges for segmentation. We conduct extensive experiments on two challenging abdominal CT datasets with eight annotated organs. Experimental results show that the proposed network outperforms several state-of-theart models, especially for the segmentation of small and complicated structures (gallbladder, esophagus, stomach, pancreas and duodenum). The code will be publicly available.

show abstract

Section: A Comparison With Existing Modelsmentioning

confidence: 94%

Section: Comparison With Existing Modelsmentioning

confidence: 99%

Section: Comparison With Existing Modelsmentioning

confidence: 99%

Section: A Comparison With Existing Modelsmentioning

confidence: 99%

See 2 more Smart Citations

Recurrent Feature Propagation and Edge Skip-Connections for Automatic Abdominal Organ Segmentation

Zhan¹,

Dong²,

Wang³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For extracting the global contextual information, the transformer elements encode the input images as a sequence of image patches. Then, the decoder performs the upsampling task needed for retrieving the precise localization [17]. The transformer architecture employs dispense convolution operators and relies on a multihead self-attention mechanism instead.…”

Section: A Swtr-unetmentioning

confidence: 99%

Joint Liver and Hepatic Lesion Segmentation in MRI using a Hybrid CNN with Transformer Layers

Hille¹,

Agrawal²,

Wybranski³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep learning-based segmentation of the liver and hepatic lesions therein steadily gains relevance in clinical practice due to the increasing incidence of liver cancer each year. Whereas various network variants with overall promising results in the field of medical image segmentation have been developed over the last years, almost all of them struggle with the challenge of accurately segmenting hepatic lesions. This lead to the idea of combining elements of convolutional and transformerbased architectures to overcome the existing limitations. This work presents a hybrid network called SWTR-Unet, consisting of a pretrained ResNet, transformer blocks as well as a common Unet-style decoder path. This network was applied to clinical liver MRI, as well as to the publicly available CT data of the liver tumor segmentation (LiTS) challenge. Additionally, multiple state-of-the-art networks were implemented and applied to both datasets, ensuring a direct comparability. Furthermore, correlation analysis and an ablation study were carried out, to investigate various influencing factors on the segmentation accuracy of our presented method.With Dice similarity scores of averaged 98 ± 2 % for liver and 81 ± 28 % lesion segmentation on the MRI dataset and 97 ± 2 % and 79 ± 25 %, respectively on the CT dataset, the proposed SWTR-Unet outperforms each of the additionally implemented state-of-the-art networks. The achieved segmentation accuracy was found to be on par with manually performed expert segmentations as indicated by interobserver variabilities for liver lesion segmentation. In conclusion, the presented method could save valuable time and resources in clinical practice.

show abstract

Materials Science at Xiamen University: A Special Issue Dedicated to the 100th Anniversary of Xiamen University

Hou

Ren

2021

Advanced Materials

View full text Add to dashboard Cite

In cardiac Magnetic Resonance Imaging (MRI) analysis, T2 mapping is a tissue quantification technology to measure water and inflammation levels and has been recognized as an important versatile index for myocardial pathologies. The T2 quantification is a highly relevant task with the myocardial segmentation but they only have been explored separately in deep learning. Here, we proposed a simultaneous dual-task network for myocardial segmentation and T2 quantification, called SQNet. It integrates Transformer and Convolutional Neural Network (CNN) components. SQNet has a T2-refine fusion decoder for quantitative analysis, incorporating global features extracted by the Transformer, and employs a segmentation decoder with multiple local region supervision to enhance segmentation accuracy. Additionally, SQNet proposes a tight coupling module that aligns and fuses CNN and Transformer branch features. The mutual promotion of the two tasks enables the SQNet to focus on the myocardium regions. The dual-task performance is validated on healthy controls (HC, N=17) and acute myocardial infarction patients (AMI, N=12). The segmentation dice of SQNet on HC/AMI is 89.3/89.2, which is higher than that (87.7/87.9) of the compared state-of-the-art segmentation method. The Pearson correlation coefficient of the T2 value on HC/AMI is 0.84 and 0.93 to the label value, indicating a strong linear correlation in myocardial T2 mapping. From the clinical diagnostic perspective, eight experienced radiologists have evaluated the image quality (Score standard: Good is 4.0 and excellent is 5.0) of SQNet on HC/AMI with the segmentation score 4.60/4.58 and the T2 quantification score 4.32/4.42, which is higher than the compared state-of-the-art segmentation score (4.50/4.44) and the T2 quantification score (3.59/4.37). Thus, SQNet provides an accurate simultaneous segmentation and quantification, providing the more accurate diagnosis of cardiac disease, such as AMI.

show abstract

TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation

Cited by 1,034 publications

References 20 publications

Recurrent Feature Propagation and Edge Skip-Connections for Automatic Abdominal Organ Segmentation

Recurrent Feature Propagation and Edge Skip-Connections for Automatic Abdominal Organ Segmentation

Joint Liver and Hepatic Lesion Segmentation in MRI using a Hybrid CNN with Transformer Layers

Materials Science at Xiamen University: A Special Issue Dedicated to the 100th Anniversary of Xiamen University

Contact Info

Product

Resources

About