Simultaneous Super-Resolution and Cross-Modality Synthesis in Magnetic Resonance Imaging

Huang, Yawen; Shao, Ling; Frangi, Alejandro F.

doi:10.1007/978-3-030-13969-8_21

Cited by 6 publications

(2 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A typical example of intensity-based methods is using image synthesis as an approach to solve sparse dictionary reconstruction, which is called dictionary learning. Huang et al (2019) introduced the cross-modality dictionary learning scheme and a patch-based globally redundant model based on sparse representations to simultaneous super-resolution and cross-modality synthesis in brain MRI. However, the atlas-based method is sensitive to alignment accuracy and segmentation precision, thus requiring time-consuming manual labeling to obtain more accurate results.…”

Section: Related Workmentioning

confidence: 99%

A layer-wise fusion network incorporating self-supervised learning for multimodal MR image synthesis

Zhou

Zou

2022

Front. Genet.

View full text Add to dashboard Cite

Magnetic resonance (MR) imaging plays an important role in medical diagnosis and treatment; different modalities of MR images can provide rich and complementary information to improve the accuracy of diagnosis. However, due to the limitations of scanning time and medical conditions, certain modalities of MR may be unavailable or of low quality in clinical practice. In this study, we propose a new multimodal MR image synthesis network to generate missing MR images. The proposed model comprises three stages: feature extraction, feature fusion, and image generation. During feature extraction, 2D and 3D self-supervised pretext tasks are introduced to pre-train the backbone for better representations of each modality. Then, a channel attention mechanism is used when fusing features so that the network can adaptively weigh different fusion operations to learn common representations of all modalities. Finally, a generative adversarial network is considered as the basic framework to generate images, in which a feature-level edge information loss is combined with the pixel-wise loss to ensure consistency between the synthesized and real images in terms of anatomical characteristics. 2D and 3D self-supervised pre-training can have better performance on feature extraction to retain more details in the synthetic images. Moreover, the proposed multimodal attention feature fusion block (MAFFB) in the well-designed layer-wise fusion strategy can model both common and unique information in all modalities, consistent with the clinical analysis. We also perform an interpretability analysis to confirm the rationality and effectiveness of our method. The experimental results demonstrate that our method can be applied in both single-modal and multimodal synthesis with high robustness and outperforms other state-of-the-art approaches objectively and subjectively.

show abstract

Section: Related Workmentioning

confidence: 99%

A layer-wise fusion network incorporating self-supervised learning for multimodal MR image synthesis

Zhou

Zou

2022

Front. Genet.

View full text Add to dashboard Cite

show abstract

“…In the end, available low resolution (LR) images should be scaled up to the higher resolution one, which is called a super resolution (SR) [13]- [15]. SR technologies have been covering many areas such as medical imaging [16], surveillance [17], and depth maps [18]. Recently, as the resolutions of televisions are increasing to 8K (7680×4320), SR methods are getting more interests [19].…”

Section: Introductionmentioning

confidence: 99%

Complexity-Reduced Super Resolution for Foveation-Based Driving Head Mounted Displays

Nam

Kang

2021

IEEE Access

View full text Add to dashboard Cite

In this paper, we propose a foveation-based super resolution (SR) algorithm to create high resolution images from low resolution inputs for virtual reality head mounted displays. Because the proposed SR algorithm is integrated in the previous foveation-based driving technology to cover the small area around the foveation point that requires high rendering quality, the overall computational complexity is substantially reduced, compared to the whole area SR. The target display has 4 times as high resolution as the input image, therefore, the proposed SR algorithm generates 4× as well as 2× SR images at the same time. To support two SR output images, small area SR, and small number of weights, we employ cropping as well as progressive and recursive framework used in the previous MS-LapSRN. We reduce the number of neurons by placing the deconvolutional layer after convolutional layers, compared to the MS-LapSRN. PSNR and SSIM performances of the proposed SR for the 4× scale are estimated as 31.152 dB and 0.935 for Set5, 26.656 dB and 0.858 for Set14, 27.138 dB and 0.830 for BSD100, and 25.078 dB and 0.836 for Urban100. For the target 8K display of 7,680×4,320, the proposed FovSR-integrated driving technology achieves the substantial reductions by 76.7 % and 99.02 % on the number of lines from 7,680 to 1,788 and the number of neurons from 24,518,246,400 to 239,541,184, respectively.

show abstract

MCMT-GAN: Multi-Task Coherent Modality Transferable GAN for 3D Brain Image Synthesis

Huang

Zheng

Cong

et al. 2020

IEEE Trans. on Image Process.

View full text Add to dashboard Cite

Simultaneous Super-Resolution and Cross-Modality Synthesis in Magnetic Resonance Imaging

Cited by 6 publications

References 43 publications

A layer-wise fusion network incorporating self-supervised learning for multimodal MR image synthesis

A layer-wise fusion network incorporating self-supervised learning for multimodal MR image synthesis

Complexity-Reduced Super Resolution for Foveation-Based Driving Head Mounted Displays

MCMT-GAN: Multi-Task Coherent Modality Transferable GAN for 3D Brain Image Synthesis

Contact Info

Product

Resources

About