AFTer-UNet: Axial Fusion Transformer UNet for Medical Image Segmentation

Yan, Xiaogang; Tang, Hao; Sun, Sumei; Ma, Haoyu; Kong, Deying; Xie, Xiaohui

doi:10.1109/wacv51458.2022.00333

Cited by 117 publications

(36 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hatamizadeh et al [2022] to the tasks of head and neck tumour segmentation using multi-modal data (i.e., CT and PET images) and compared their results with traditional CNN-based approaches. Yan et al Yan et al [2022] also employed a U-Net-based structure for the task of multi-organ segmentation in 3D medical image data, however, with slightly different changes: in their approach, they used a CNN-encoder and CNN-decoder with a Transformer model in between to fuse contextual information in the neighboring image slices.…”

Section: Multi-task Segmentationmentioning

confidence: 99%

A survey on attention mechanisms for medical applications: are we moving towards better algorithms?

Gonçalves¹,

Rio-Torto²,

Teixeira³

et al. 2022

Preprint

View full text Add to dashboard Cite

The increasing popularity of attention mechanisms in deep learning algorithms for computer vision and natural language processing made these models attractive to other research domains. In healthcare, there is a strong need for tools that may improve the routines of the clinicians and the patients. Naturally, the use of attention-based algorithms for medical applications occurred smoothly. However, being healthcare a domain that depends on high-stake decisions, the scientific community must ponder if these high-performing algorithms fit the needs of medical applications. With this motto, this paper extensively reviews the use of attention mechanisms in machine learning (including Transformers) for several medical applications. This work distinguishes itself from its predecessors by proposing a critical analysis of the claims and potentialities of attention mechanisms presented in the literature through an experimental case study on medical image classification with three different use cases. These experiments focus on the integrating process of attention mechanisms into established deep learning architectures, the analysis of their predictive power, and a visual assessment of their saliency maps generated by post-hoc explanation methods. This paper concludes with a critical analysis of the claims and potentialities presented in the literature about attention mechanisms and proposes future research lines in medical applications that may benefit from these frameworks.

show abstract

Section: Multi-task Segmentationmentioning

confidence: 99%

A survey on attention mechanisms for medical applications: are we moving towards better algorithms?

Gonçalves¹,

Rio-Torto²,

Teixeira³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The proposed model is composed of a down-sample part, an up-sample part, and a connection part. The UTNet [110] 2021 MRI left ventricle, right ventricle, left ventricular myocardium [111] MRA-TUNet [112] 2022 MRI left ventricle, right ventricle, left ventricular myocardium, left atrium cardiac disease [113], atrial fibrillation [114] HybridCTrm [115] 2021 MRI brain [116], neurodevelopmental disorders [117] consistency-based co-segmentation [118] 2021 MRI right ventricle [119] TransConver [120] 2022 MRI brain brain tumor [121,122,123] UTransNet [124] 2022 MRI brain stroke [129] TransBTS [125] 2021 MRI brain brain tumor [121,122,123] METrans [126] 2022 MRI brain stroke [130], ischemic stroke lesion [131], schemic stroke lesion [132] SwinBTS [127] 2022 MRI brain brain tumor [121,123,133,134] BTSwin-Unet [128] 2022 MRI brain brain tumor [121,122] CVT-Vnet [135] 2022 CT head, neck organs at risk [136] CoTr [137] 2021 CT abdomen colorectal cancer, ventral hernia [138] transformer-UNet [139] 2021 CT lung [140] AFTer-UNet [141] 2022 CT abdomen, thorax [142], organs at risk…”

Section: Segmentationmentioning

confidence: 99%

Recent Progress in Transformer-based Medical Image Analysis

Liu¹,

Shen²

2022

Preprint

View full text Add to dashboard Cite

The transformer has dominated the natural language processing (NLP) field for a long time. Recently, the transformer-based method has been adopted into the computer vision (CV) field and shows promising results. As an important branch of the CV field, medical image analysis joins the wave of the transformer-based method rightfully. In this review, we illustrate the principle of the attention mechanism, and the detailed structures of the transformer, and depict how the transformer is adopted into medical image analysis. We organize the transformer-based medical image analysis applications in a sequence of different tasks, including classification, segmentation, synthesis, registration, localization, detection, captioning, and denoising. For the mainstream classification and segmentation tasks, we further divided the corresponding works based on different medical imaging modalities. The datasets corresponding to the related works are also organized. We include thirteen modalities and more than twenty objects in our work.

show abstract

“…However, the convolution kernel usually has a limited receptive field and thus cannot capture long-range dependencies, which are essential for MR image-to-image translation. Nowadays, vision transformer ( 20 ) is capable of modeling global interactions between contexts and has promising performance in MRI restoration ( 21 , 22 ), segmentation ( 23 , 24 ), and registration ( 25 , 26 ). Nevertheless, vision transformers for image restoration need to divide the input image into small patches of fixed size, which may introduce border artifacts around each small patch in the restored images.…”

Section: Introductionmentioning

confidence: 99%

Swin transformer-based GAN for multi-modal medical image translation

et al. 2022

View full text Add to dashboard Cite

Medical image-to-image translation is considered a new direction with many potential applications in the medical field. The medical image-to-image translation is dominated by two models, including supervised Pix2Pix and unsupervised cyclic-consistency generative adversarial network (GAN). However, existing methods still have two shortcomings: 1) the Pix2Pix requires paired and pixel-aligned images, which are difficult to acquire. Nevertheless, the optimum output of the cycle-consistency model may not be unique. 2) They are still deficient in capturing the global features and modeling long-distance interactions, which are critical for regions with complex anatomical structures. We propose a Swin Transformer-based GAN for Multi-Modal Medical Image Translation, named MMTrans. Specifically, MMTrans consists of a generator, a registration network, and a discriminator. The Swin Transformer-based generator enables to generate images with the same content as source modality images and similar style information of target modality images. The encoder part of the registration network, based on Swin Transformer, is utilized to predict deformable vector fields. The convolution-based discriminator determines whether the target modality images are similar to the generator or from the real images. Extensive experiments conducted using the public dataset and clinical datasets showed that our network outperformed other advanced medical image translation methods in both aligned and unpaired datasets and has great potential to be applied in clinical applications.

show abstract

AFTer-UNet: Axial Fusion Transformer UNet for Medical Image Segmentation

Cited by 117 publications

References 27 publications

A survey on attention mechanisms for medical applications: are we moving towards better algorithms?

A survey on attention mechanisms for medical applications: are we moving towards better algorithms?

Recent Progress in Transformer-based Medical Image Analysis

Swin transformer-based GAN for multi-modal medical image translation

Contact Info

Product

Resources

About