HiFormer: Hierarchical Multi-scale Representations Using Transformers for Medical Image Segmentation

Heidari, Moein; Kazerouni, Amirhossein; Soltany, Milad; Azad, Reza; Aghdam, Ehsan Khodapanah; Cohen‐Adad, Julien; Merhof, Dorit

doi:10.48550/arxiv.2207.08518

Cited by 4 publications

(3 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Image segmentation is a vital task in computer vision, which investigates simplifying the complexity of the image by decomposing an image into multiple meaningful image segments [103,104]. Specifically, it facilitates medical analysis by providing beneficial information about anatomy-related areas.…”

Section: Segmentationmentioning

confidence: 99%

Diffusion Models for Medical Image Analysis: A Comprehensive Survey

Kazerouni¹,

Aghdam²,

Heidari³

et al. 2022

Preprint

View full text Add to dashboard Cite

Denoising diffusion models, a class of generative models, have garnered immense interest lately in various deep-learning problems. A diffusion probabilistic model defines a forward diffusion stage where the input data is gradually perturbed over several steps by adding Gaussian noise and then learns to reverse the diffusion process to retrieve the desired noise-free data from noisy data samples. Diffusion models are widely appreciated for their strong mode coverage and quality of the generated samples in spite of their known computational burdens. Capitalizing on the advances in computer vision, the field of medical imaging has also observed a growing interest in diffusion models. With the aim of helping the researcher navigate this profusion, this survey intends to provide a comprehensive overview of diffusion models in the discipline of medical image analysis. Specifically, we start with an introduction to the solid theoretical foundation and fundamental concepts behind diffusion models and the three generic diffusion modeling frameworks, namely, diffusion probabilistic models, noise-conditioned score networks, and stochastic differential equations. Then, we provide a systematic taxonomy of diffusion models in the medical domain and propose a multi-perspective categorization based on their application, imaging modality, organ of interest, and algorithms. To this end, we cover extensive applications of diffusion models in the medical domain, including segmentation, anomaly detection, image-to-image translation, 2/3D generation, reconstruction, denoising, and other medically-related challenges. Furthermore, we emphasize the practical use case of some selected approaches, and then we discuss the limitations of the diffusion models in the medical domain and propose several directions to fulfill the demands of this field. Finally, we gather the overviewed studies with their available open-source implementations at our GitHub 1 . We aim to update the relevant latest papers within it regularly.

show abstract

Section: Segmentationmentioning

confidence: 99%

Diffusion Models for Medical Image Analysis: A Comprehensive Survey

Kazerouni¹,

Aghdam²,

Heidari³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Yet, a persistent challenge remains: Vision transformers, while powerful, often overlook some critical low-level features. This observation has spurred the development of hybrid models like HiFormer (Heidari et al 2023) and TransUnet, which aim to merge the strengths of both CNNs and transformers. However, seamlessly blending features and leveraging multi-scale information remains a complex task for these hybrid models.…”

Section: Introductionmentioning

confidence: 99%

DDViT: Double-Level Fusion Domain Adapter Vision Transformer (Student Abstract)

Sun,

Sheng

2024

AAAI

View full text Add to dashboard Cite

With the help of Vision transformers (ViTs), medical image segmentation was able to achieve outstanding performance. In particular, they overcome the limitation of convolutional neural networks (CNNs) which rely on local receptive fields. ViTs use self-attention mechanisms to consider relationships between all image pixels or patches simultaneously. However, they require large datasets for training and did not perform well on capturing low-level features. To that end, we propose DDViT, a novel ViT model that unites a CNN to alleviate data-hunger for medical image segmentation with two multi-scale feature representations. Significantly, our approach incorporates a ViT with a plug-in domain adapter (DA) with Double-Level Fusion (DLF) technique, complemented by a mutual knowledge distillation paradigm, facilitating the seamless exchange of knowledge between a universal network and specialized domain-specific network branches. The DLF framework plays a pivotal role in our encoder-decoder architecture, combining the innovation of the TransFuse module with a robust CNN-based encoder. Extensive experimentation across diverse medical image segmentation datasets underscores the remarkable efficacy of DDViT when compared to alternative approaches based on CNNs and Transformer-based models.

show abstract

“…Automatic and accurate medical image segmentation, which consists of automated delineation of anatomical structures and other regions of interest (ROIs), plays an integral role in the assessment of computer-aided diagnosis (CAD) [9,23,19,17,3,7]. As a flagship of deep learning, convolutional neural networks (CNNs) have scattered existing contributions in various medical image segmentation tasks for many years [31,28,5,4,6].…”

Section: Introductionmentioning

confidence: 99%

TransDeepLab: Convolution-Free Transformer-Based DeepLab v3+ for Medical Image Segmentation

Azad

Heidari

Shariatnia

et al. 2022

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) have been the de facto standard in a diverse set of computer vision tasks for many years. Especially, deep neural networks based on seminal architectures such as U-shaped model with skip-connections or atrous convolution with pyramid pooling have been tailored to a wide range of medical image analysis tasks. The main advantage of such architectures is that they are prone to detaining versatile local features. However, as a general consensus, CNNs fail to capture long-range dependencies and spatial correlations due to the intrinsic property of confined receptive field size of convolution operations. Alternatively, Transformer, profiting from global information modeling that stems from the self-attention mechanism, has recently attained remarkable performance in natural language processing and computer vision. Nevertheless, previous studies prove that both local and global features are critical for a deep model in dense prediction, such as segmenting complicated structures with disparate shapes and configurations. To this end, this paper proposes TransDeepLab, a novel DeepLab-like pure Transformer for medical image segmentation. Specifically, we exploit hierarchical Swin-Transformer with shifted windows to extend the DeepLabv3 and model the Atrous Spatial Pyramid Pooling (ASPP) module. A thorough search of the relevant literature yielded that we are the first to model the seminal DeepLab model with a pure Transformer-based model. Extensive experiments on various medical image segmentation tasks verify that our approach performs superior or on par with most contemporary works on an amalgamation of Vision Transformer and CNN-based methods, along with a significant reduction of model complexity. The codes and trained models are publicly available at github..

show abstract

HiFormer: Hierarchical Multi-scale Representations Using Transformers for Medical Image Segmentation

Cited by 4 publications

References 42 publications

Diffusion Models for Medical Image Analysis: A Comprehensive Survey

Diffusion Models for Medical Image Analysis: A Comprehensive Survey

DDViT: Double-Level Fusion Domain Adapter Vision Transformer (Student Abstract)

TransDeepLab: Convolution-Free Transformer-Based DeepLab v3+ for Medical Image Segmentation

Contact Info

Product

Resources

About