BTSwin-Unet: 3D U-shaped Symmetrical Swin Transformer-based Network for Brain Tumor Segmentation with Self-supervised Pre-training

Liang, Junjie; Yang, Cihui; Zhong, Jingting; Yang, Xiaoli

doi:10.1007/s11063-022-10919-1

Cited by 26 publications

(20 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Shifted Window (SWIN) transformer [ 49 ] has recently become a popular choice for image-based classification tasks. Therefore, the most recent studies [ 27 - 29 , 34 , 39 , 45 ] reported using SWIN transformers in their models. Some of the studies [ 28 , 29 , 36 , 38 , 40 , 41 ] incorporated the transformers module within the encoder or decoder or both modules of the UNet-like architectures.…”

Section: Resultsmentioning

confidence: 99%

Artificial Intelligence–Based Methods for Integrating Local and Global Features for Brain Cancer Imaging: Scoping Review

Ali,

Qureshi,

Shah

2023

JMIR Med Inform

View full text Add to dashboard Cite

Background Transformer-based models are gaining popularity in medical imaging and cancer imaging applications. Many recent studies have demonstrated the use of transformer-based models for brain cancer imaging applications such as diagnosis and tumor segmentation. Objective This study aims to review how different vision transformers (ViTs) contributed to advancing brain cancer diagnosis and tumor segmentation using brain image data. This study examines the different architectures developed for enhancing the task of brain tumor segmentation. Furthermore, it explores how the ViT-based models augmented the performance of convolutional neural networks for brain cancer imaging. Methods This review performed the study search and study selection following the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. The search comprised 4 popular scientific databases: PubMed, Scopus, IEEE Xplore, and Google Scholar. The search terms were formulated to cover the interventions (ie, ViTs) and the target application (ie, brain cancer imaging). The title and abstract for study selection were performed by 2 reviewers independently and validated by a third reviewer. Data extraction was performed by 2 reviewers and validated by a third reviewer. Finally, the data were synthesized using a narrative approach. Results Of the 736 retrieved studies, 22 (3%) were included in this review. These studies were published in 2021 and 2022. The most commonly addressed task in these studies was tumor segmentation using ViTs. No study reported early detection of brain cancer. Among the different ViT architectures, Shifted Window transformer–based architectures have recently become the most popular choice of the research community. Among the included architectures, UNet transformer and TransUNet had the highest number of parameters and thus needed a cluster of as many as 8 graphics processing units for model training. The brain tumor segmentation challenge data set was the most popular data set used in the included studies. ViT was used in different combinations with convolutional neural networks to capture both the global and local context of the input brain imaging data. Conclusions It can be argued that the computational complexity of transformer architectures is a bottleneck in advancing the field and enabling clinical transformations. This review provides the current state of knowledge on the topic, and the findings of this review will be helpful for researchers in the field of medical artificial intelligence and its applications in brain cancer.

show abstract

Section: Resultsmentioning

confidence: 99%

Artificial Intelligence–Based Methods for Integrating Local and Global Features for Brain Cancer Imaging: Scoping Review

Ali,

Qureshi,

Shah

2023

JMIR Med Inform

View full text Add to dashboard Cite

show abstract

“…The proposed work inserts a transformer encoder between the encoder and the decoder and the skip connection is implemented between the encoder and decoder at different scales. Similar works including METrans proposed by Wang et al [126], SwinBTS developed by Jiang and colleagues [127], and BTSwin-Unet implemented by Liang et al [128].…”

Section: Segmentationmentioning

confidence: 87%

“…The proposed model is composed of a down-sample part, an up-sample part, and a connection part. The UTNet [110] 2021 MRI left ventricle, right ventricle, left ventricular myocardium [111] MRA-TUNet [112] 2022 MRI left ventricle, right ventricle, left ventricular myocardium, left atrium cardiac disease [113], atrial fibrillation [114] HybridCTrm [115] 2021 MRI brain [116], neurodevelopmental disorders [117] consistency-based co-segmentation [118] 2021 MRI right ventricle [119] TransConver [120] 2022 MRI brain brain tumor [121,122,123] UTransNet [124] 2022 MRI brain stroke [129] TransBTS [125] 2021 MRI brain brain tumor [121,122,123] METrans [126] 2022 MRI brain stroke [130], ischemic stroke lesion [131], schemic stroke lesion [132] SwinBTS [127] 2022 MRI brain brain tumor [121,123,133,134] BTSwin-Unet [128] 2022 MRI brain brain tumor [121,122] CVT-Vnet [135] 2022 CT head, neck organs at risk [136] CoTr [137] 2021 CT abdomen colorectal cancer, ventral hernia [138] transformer-UNet [139] 2021 CT lung [140] AFTer-UNet [141] 2022 CT abdomen, thorax [142], organs at risk…”

Section: Segmentationmentioning

confidence: 99%

Recent Progress in Transformer-based Medical Image Analysis

Liu¹,

Shen²

2022

Preprint

View full text Add to dashboard Cite

The transformer has dominated the natural language processing (NLP) field for a long time. Recently, the transformer-based method has been adopted into the computer vision (CV) field and shows promising results. As an important branch of the CV field, medical image analysis joins the wave of the transformer-based method rightfully. In this review, we illustrate the principle of the attention mechanism, and the detailed structures of the transformer, and depict how the transformer is adopted into medical image analysis. We organize the transformer-based medical image analysis applications in a sequence of different tasks, including classification, segmentation, synthesis, registration, localization, detection, captioning, and denoising. For the mainstream classification and segmentation tasks, we further divided the corresponding works based on different medical imaging modalities. The datasets corresponding to the related works are also organized. We include thirteen modalities and more than twenty objects in our work.

show abstract

“…Of course, the development of 3D Transformer-based fracture recognition technology is also one of our future research directions. Currently, in the research of 3D medical image semantic segmentation, some scholars have developed 3D Transformer models, which can provide references for our future research on 3D Transformer-based fault detection (Hatamizadeh et al, 2022;Liang et al, 2022). However, we need to develop a 3D Transformer model suitable for fault recognition in seismic data according to the characteristics of seismic data.…”

Section: Discussionmentioning

confidence: 99%

Transformer assisted dual U-net for seismic fault detection

et al. 2023

View full text Add to dashboard Cite

Automatic seismic fault identification for seismic data is essential for oil and gas resource exploration. The traditional manual method cannot accommodate the needs of processing massive seismic data. With the development of artificial intelligence technology, deep learning techniques based on pattern recognition have become a popular research area for seismic fault identification. Despite the progress made with U-shaped neural networks (Unet), they still fall short in meeting the stringent requirements of fault prediction in complex structures. We propose a novel approach by combining a standard Unet with a transformer Unet to create a parallel dual Unet model, called Dual Unet with Transformer. To improve the accuracy of fault prediction, we compare six loss functions (including Binary Cross Entropy loss, Dice coefficient loss, Tversky loss, Local Tversky loss, Multi-scale Structural Similarity and Intersection over Union loss) using synthetic data, based on three evolution metrics involving Dice coefficient, Sensitivity and Specificity, find that the binary cross entropy loss function is the most robust one. An example comparing the prediction performance of different Unet models on synthetic data demonstrates the superior performance of our Dual Unet model, verifying the practical application value. To further validate the practical feasibility of our proposed method, we use real seismic data with a complex fault system and find that our proposed model is more accurate in predicting the fault system compared to well-developed Unet models such as the classical Unet and classical coherence cube algorithm, without transfer learning. This confirms the potential for wide-scale application of our proposed model.

show abstract

BTSwin-Unet: 3D U-shaped Symmetrical Swin Transformer-based Network for Brain Tumor Segmentation with Self-supervised Pre-training

Cited by 26 publications

References 24 publications

Artificial Intelligence–Based Methods for Integrating Local and Global Features for Brain Cancer Imaging: Scoping Review

Artificial Intelligence–Based Methods for Integrating Local and Global Features for Brain Cancer Imaging: Scoping Review

Recent Progress in Transformer-based Medical Image Analysis

Transformer assisted dual U-net for seismic fault detection

Contact Info

Product

Resources

About