Joint Reconstruction and Segmentation of 7T-like MR Images from 3T MRI Based on Cascaded Convolutional Neural Networks

Abstract

7T MRI scanner provides MR images with higher resolution and better contrast than 3T MR scanners. This helps many medical analysis tasks, including tissue segmentation. However, currently there is a very limited number of 7T MRI scanners worldwide. This motivates us to propose a novel image post-processing framework that can jointly generate high-resolution 7T-like images and their corresponding high-quality 7T-like tissue segmentation maps, solely from the routine 3T MR images. Our proposed framework comprises two parallel components, namely (1) reconstruction and (2) segmentation. The reconstruction component includes the multi-step cascaded convolutional neural networks (CNNs) that map the input 3T MR image to a 7T-like MR image, in terms of both resolution and contrast. Similarly, the segmentation component involves another paralleled cascaded CNNs, with a different architecture, to generate high-quality segmentation maps. These cascaded feedbacks between the two designed paralleled CNNs allow both tasks to mutually benefit from each another when learning the respective reconstruction and segmentation mappings. For evaluation, we have tested our framework on 15 subjects (with paired 3T and 7T images) using a leave-one-out cross-validation. The experimental results show that our estimated 7T-like images have richer anatomical details and better segmentation results, compared to the 3T MRI. Furthermore, our method also achieved better results in both reconstruction and segmentation tasks, compared to the state-of-the-art methods.

Improving resolution of MR images with an adversarial network incorporating images with different contrast

“…In recent studies, deep learning algorithms, especially convolutional neural networks (CNNs), have been applied to reconstruct MR images from the incoherently down‐sampled data or to produce 7T‐like high resolution (HR) images from 3T MR images . Bahrami et al.…”

“…Bahrami et al. demonstrated that CNNs can produce a 7T‐like HR image from a 3T MR image as input with real 7T MR images as label . Wang et al.…”

“…5 In recent studies, deep learning algorithms, especially convolutional neural networks (CNNs), have been applied to reconstruct MR images from the incoherently down-sampled data [6][7][8][9][10] or to produce 7T-like high resolution (HR) images from 3T MR images. 11,12 Bahrami et al demonstrated that CNNs can produce a 7T-like HR image from a 3T MR image as input with real 7T MR images as label. 11,12 Wang et al proposed a framework combing CNNs with online CS-MRI methods for effective imaging.…”

“…11,12 Bahrami et al demonstrated that CNNs can produce a 7T-like HR image from a 3T MR image as input with real 7T MR images as label. 11,12 Wang et al proposed a framework combing CNNs with online CS-MRI methods for effective imaging. 6 Also, CNNs alone can reconstruct images from the down-sampled data effectively.…”

Prospects of deep learning for medical imaging

“…Many studies adopt multimodal imaging and thus registration is necessary to extract spatially aligned features in those studies. Registration requires a long computational time as it involves a high degree of freedom (DOF) optimiza- MRI CT Image generation [69] Diffusion MRI Diffusion MRI Image enhancement [70] MRI (3T) MRI (7T) Image reconstruction [71] Deep learning for medical imaging tion problem. The CNN-based approaches could be used as an alternative for the difficult registration problem by finding landmarks or control points through CNN architecture.…”