Multimodal Transformer for Accelerated MR Imaging

Feng, Chun-Mei; Yan, Yunlu; Chen, Geng; Xu, Yong; Hu, Ying; Shao, Ling; Fu, Huazhu

doi:10.1109/tmi.2022.3180228

Cited by 70 publications

(38 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Performance evaluation. We compare our methods with other baseline deep learning methods in three conventions of MRI reconstruction: image-domain [24,34,6,9,13], dual-domain [24,34,33] and reference-protocol-guided dual-domain reconstruction [29,6,34,33,19]. All reference-guided methods are self-implemented besides Du-DoRNet and examined without considering multi-modal fusion modules for controlled backbone comparisons.…”

Section: Settings and Resultsmentioning

confidence: 99%

See 1 more Smart Citation

DuDoRNeXt: A hybrid model for dual-domain undersampled MRI reconstruction

Gao¹,

Zhou²

2023

Preprint

View full text Add to dashboard Cite

Undersampled MRI reconstruction is crucial for accelerating clinical scanning procedures. Recent deep learning methods for MRI reconstruction adopt CNN or ViT as backbone, which lack in utilizing the complementary properties of CNN and ViT. In this paper, we propose DuDoRNeXt, whose backbone hybridizes CNN and ViT in an domainspecific, intra-stage way. Besides our hybrid vertical layout design, we introduce domain-specific modules for dual-domain reconstruction, namely image-domain parallel local detail enhancement and k-space global initialization. We evaluate different conventions of MRI reconstruction including image-domain, k-space-domain, and dual-domain reconstruction with a reference protocol on the IXI dataset and an in-house multicontrast dataset. DuDoRNeXt achieves significant improvements over competing deep learning methods.

show abstract

Section: Settings and Resultsmentioning

confidence: 99%

“…Recently, a growing number of works [21,29,6,34,33,19] utilize a fast-to-acquire fully-sampled auxiliary MRI protocol to guide the reconstruction of a slow protocol.…”

Section: Undersampled Mri Reconstructionmentioning

confidence: 99%

DuDoRNeXt: A hybrid model for dual-domain undersampled MRI reconstruction

Gao¹,

Zhou²

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…With some under‐sampled and noisy input images, deep learning can reconstruct the ideal MRI images. Among these methods, both CNN‐based networks and Transformer‐based model bear its own advantages and drawbacks 170,171,174,175,177,179,181,184 . made corresponding adjustments to some part of MRI, including k‐space and sampling to promote the speed of MRI.…”

Section: Medical Image Reconstructionmentioning

confidence: 99%

“…Among these methods, both CNN-based networks and Transformer-based model bear its own advantages and drawbacks. 170,171,174,175,177,179,181,184 made corresponding adjustments to some part of MRI, including k-space and sampling to promote the speed of MRI. The scan of MRI demands much time to generate the complete K-space matrices.…”

Section: Accelerated Reconstructionmentioning

confidence: 99%

Recent advances of Transformers in medical image analysis: A comprehensive review

Xia

Wang

2023

MedComm – Future Medicine

View full text Add to dashboard Cite

Recent works have shown that Transformer's excellent performances on natural language processing tasks can be maintained on natural image analysis tasks. However, the complicated clinical settings in medical image analysis and varied disease properties bring new challenges for the use of Transformer. The computer vision and medical engineering communities have devoted significant effort to medical image analysis research based on Transformer with especial focus on scenario-specific architectural variations.In this paper, we comprehensively review this rapidly developing area by covering the latest advances of Transformer-based methods in medical image analysis of different settings. We first give introduction of basic mechanisms of Transformer including implementations of selfattention and typical architectures. The important research problems in various medical image data modalities, clinical visual tasks, organs and diseases are then reviewed systemically. We carefully collect 276 very recent works and 76 public medical image analysis datasets in an organized structure. Finally, discussions on open problems and future research directions are also provided. We expect this review to be an up-to-date roadmap and serve as a reference source in pursuit of boosting the development of medical image analysis field.

show abstract

“…S INGLE-Image Super-Resolution (SISR) technique aims to generate natural and realistic textures in a high-resolution (HR) image by only utilizing its deteriorated low-resolution (LR) counterpart. SISR has been a hotspot for study in academic and industry thanks to its many applications, including remote sensing imaging [1] [2] [3] [4], medical imaging [5] [6] [7] and face recognition [8]. SISR is a classic ill-posed problem since numerous distinct high-resolution images can be mapped to the same low-resolution image, which poses a significant challenge to restoration task.…”

Section: Introductionmentioning

confidence: 99%

Multi-Scale Fast Fourier Transform Based Attention Network for Remote-Sensing Image Super-Resolution

Wang¹,

Zhao²,

Chen

2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Recently, with the rise and progress of convolutional neural networks (CNNs), CNN-based remote sensing image super-resolution (RSSR) methods have gained considerable advancement and showed great power for image reconstruction tasks. However, most of these methods cannot handle well the enormous number of objects with different scales contained in remote sensing images and thus limits super-resolution performance. To address these issues, we propose a Multi-Scale Fast Fourier Transform (FFT) based Attention Network (MSFFTAN) which employs a multi-input U-shape structure as backbone for accurate remote sensing image super-resolution. Specifically, we carefully design an FFT-based residual block consisting of an image domain branch and a Fourier domain branch to extract local details and global structures simultaneously. In addition, a Local-Global Channel Attention Block (LGCAB) is developed to further enhance the reconstruction ability of small targets. Finally, we present a Branch Gated Selective Block (BGSB) to adaptively explore and aggregate features from multiple scales and depths. Extensive experiments on two publicly datasets have demonstrated the superiority of MSFFTAN over the state-of-theart (SOAT) approaches in aspects of both quantitative metrics and visual quality. The peak signal-to-noise ratio of our network is 1.5 dB higher than the SOAT method on the UCMerced LandUse with downscaling factor 2.

show abstract

Multimodal Transformer for Accelerated MR Imaging

Cited by 70 publications

References 63 publications

DuDoRNeXt: A hybrid model for dual-domain undersampled MRI reconstruction

DuDoRNeXt: A hybrid model for dual-domain undersampled MRI reconstruction

Recent advances of Transformers in medical image analysis: A comprehensive review

Multi-Scale Fast Fourier Transform Based Attention Network for Remote-Sensing Image Super-Resolution

Contact Info

Product

Resources

About