SWTRU: Star-shaped Window Transformer Reinforced U-Net for medical image segmentation

Zhang, Jianyi; Liu, Yong; Wu, Qihang; Wang, Yongpan; Liu, Yuhai; Xu, Xianchong; Song, Bo

doi:10.1016/j.compbiomed.2022.105954

Cited by 27 publications

(13 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As an alternative form of deep learning, transformers have already performed well in natural language processing 44 . In computer vision, transformer‐based architectures have also outperformed CNN‐based architectures in many classification tasks 45–47 . CNN is inherently designed to exploit the spatial locality in images due to the local‐receptive fields of convolutional filters.…”

Section: Methodsmentioning

confidence: 99%

“…44 In computer vision, transformer-based architectures have also outperformed CNN-based architectures in many classification tasks. [45][46][47] CNN is inherently designed to exploit the spatial locality in images due to the local-receptive fields of convolutional filters. Convolutional layers scan the input image with window filters, enabling the model to learn hierarchical representations of local patterns.…”

Section: Vision Transformermentioning

confidence: 99%

See 1 more Smart Citation

Predictive stroke risk model with vision transformer‐based Doppler features

Lo,

Hung

2023

Medical Physics

View full text Add to dashboard Cite

BackgroundAcute stroke is the leading cause of death and disability globally, with an estimated 16 million cases each year. The progression of carotid stenosis reduces blood flow to the intracranial vasculature, causing stroke. Early recognition of ischemic stroke is crucial for disease treatment and management.PurposeA computer‐aided diagnosis (CAD) system was proposed in this study to rapidly evaluate ischemic stroke in carotid color Doppler (CCD).MethodsBased on the ground truth from the clinical examination report, the vision transformer (ViT) features extracted from all CCD images (513 stroke and 458 normal images) were combined in machine learning classifiers to generate the likelihood of ischemic stroke for each image. The pretrained weights from ImageNet reduced the time‐consuming training process. The accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve were calculated to evaluate the stroke prediction model. The chi‐square test, DeLong test, and Bonferroni correction for multiple comparisons were applied to deal with the type‐I error. Only p values equal to or less than 0.00125 were considered to be statistically significant.ResultsThe proposed CAD system achieved an accuracy of 89%, a sensitivity of 94%, a specificity of 84%, and an area under the receiver operating characteristic curve of 0.95, outperforming the convolutional neural networks AlexNet (82%, p < 0.001), Inception‐v3 (78%, p < 0.001), ResNet101 (84%, p < 0.001), and DenseNet201 (85%, p < 0.01). The computational time in model training was only 30 s, which would be efficient and practical in clinical use.ConclusionsThe experiment shows the promising use of CCD images in stroke estimation. Using the pretrained ViT architecture, the image features can be automatically and efficiently generated without human intervention. The proposed CAD system provides a rapid and reliable suggestion for diagnosing ischemic stroke.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Vision Transformermentioning

confidence: 99%

Predictive stroke risk model with vision transformer‐based Doppler features

Lo,

Hung

2023

Medical Physics

View full text Add to dashboard Cite

show abstract

“…This outcome pushed the number of layers in the neural network to 201, as shown in figure 2(a).The transformer(Vaswani et al 2017), another deep learning architecture originally employed for natural language processing, was recently developed into the vision transformer (ViT). In experiments, ViT indeed achieved higher accuracy than ResNet(Sengar et al 2022, Xin et al 2022, Zhang et al 2022b. ViT divides images into a series of patches to consider global correlations in feature extraction(Dosovitskiy et al 2020).…”

mentioning

confidence: 92%

Interactive content-based image retrieval with deep learning for CT abdominal organ recognition

Lo,

Wang,

Hung

2024

Phys. Med. Biol.

View full text Add to dashboard Cite

Objective: Recognizing the most relevant seven organs in an abdominal computed tomography (CT) slice requires sophisticated knowledge. This study proposed automatically extracting relevant features and applying them in a content-based image retrieval (CBIR) system to provide similar evidence for clinical use.Approach: A total of 2827 abdominal CT slices, including 638 liver, 450 stomach, 229 pancreas, 442 spleen, 362 right kidney, 424 left kidney and 282 gallbladder tissues, were collected to evaluate the proposed CBIR in the present study. Upon fine-tuning, high-level features used to automatically interpret the differences among the seven organs were extracted via deep learning architectures, including DenseNet, Vision Transformer (ViT), and Swin Transformer v2 (SwinViT). Three images with different annotations were employed in the classification and query.Main results: The resulting performances included the classification accuracy (94%-99%) and retrieval result (0.98-0.99). Considering global features and multiple resolutions, SwinViT performed better than ViT. ViT also benefited from a better receptive field to outperform DenseNet. Additionally, the use of hole images can obtain almost perfect results regardless of which deep learning architectures are used.Significance: The experiment showed that using pretrained deep learning architectures and fine-tuning with enough data can achieve successful recognition of seven abdominal organs. The CBIR system can provide more convincing evidence for recognizing abdominal organs via similarity measurements, which could lead to additional possibilities in clinical practice.

show abstract

“…Affected by the great success of U‐Net in medical image segmentation, U‐Net++ 17 and ResUNet++ 18 are also suitable for medical image segmentation and have achieved good results. Nowadays, most researchers achieve the purpose of improving network performance by adding or modifying various modules 19–25 . PraNet 19 proposed a parallel reverse attention network structure for segmentation using the polyp area and boundary information as cues.…”

Section: Introductionmentioning

confidence: 99%

“…Nowadays, most researchers achieve the purpose of improving network performance by adding or modifying various modules. [19][20][21][22][23][24][25] PraNet 19 proposed a parallel reverse attention network structure for segmentation using the polyp area and boundary information as cues. And aggregates the features with high-level semantics using a parallel partial decoder (PPD).…”

mentioning

confidence: 99%

TACT: Text attention based CNN‐Transformer network for polyp segmentation

Zhao,

Li,

Hua

2023

Int J Imaging Syst Tech

View full text Add to dashboard Cite

Colorectal cancer (CRC) has been one of the top three disease in the world in terms of incidence for many years. Therefore, how to prevent and treat CRC has become a topic of concern for an increasing number of people, and colonoscopy is the most effective detection method in polyp examination. According to studies, 90% of CRC is caused by adenomatous polyps of the large intestine. In clinical practice, the diversity of polyps' size, number, and shape and the unclear boundary between polyps and colon folds can reduce the operator's accuracy of polyps segmentation and lead to a higher rate of missed diagnosis. To better address the inaccurate segmentation or high miss rate due to the above factors, we propose a text attention‐based CNN‐Transformer network for polyp segmentation (TACT) network to process the images in a way that minimizes operator subjectivity and miss rate. The network is based on the CNN‐Transformer structure, and on this basis, a fully attention progressive sampling module is added to more accurately divide the polyp boundary. Moreover, an auxiliary text classification task was added to focus on polyp size and number features in the form of text attention, which more effectively copes with the segmentation tasks of different sizes and different numbers of polyps. After comparing with multiple state‐of‐the‐art segmentation methods in four challenging datasets, our proposed TACT improves segmentation accuracy for polyps of different sizes in different datasets.

show abstract

SWTRU: Star-shaped Window Transformer Reinforced U-Net for medical image segmentation

Cited by 27 publications

References 9 publications

Predictive stroke risk model with vision transformer‐based Doppler features

Predictive stroke risk model with vision transformer‐based Doppler features

Interactive content-based image retrieval with deep learning for CT abdominal organ recognition

TACT: Text attention based CNN‐Transformer network for polyp segmentation

Contact Info

Product

Resources

About