Multiscale brain MRI super-resolution using deep 3D convolutional networks

Abstract

The purpose of super-resolution approaches is to overcome the hardware limitations and the clinical requirements of imaging procedures by reconstructing high-resolution images from low-resolution acquisitions using post-processing methods. Super-resolution techniques could have strong impacts on structural magnetic resonance imaging when focusing on cortical surface or fine-scale structure analysis for instance. In this paper, we study deep three-dimensional convolutional neural networks for the super-resolution of brain magnetic resonance imaging data. First, our work delves into the relevance of several factors in the performance of the purely convolutional neural network-based techniques for the monomodal super-resolution: optimization methods, weight initialization, network depth, residual learning, filter size in convolution layers, number of the filters, training patch size and number of training subjects. Second, our study also highlights that one single network can efficiently handle multiple arbitrary scaling factors based on a multiscale training approach. Third, we further extend our super-resolution networks to the multimodal super-resolution using intermodality priors. Fourth, we investigate the impact of transfer learning skills onto super-resolution performance in terms of generalization among different datasets. Lastly, the learnt models are used to enhance real clinical low-resolution images. Results tend to demonstrate the potential of deep neural networks with respect to practical medical image applications.

Convolutional Neural Networks With Intermediate Loss for 3D Super-Resolution of CT and MRI Scans

“…We also present the 3D super-resolution results in Table 6. The 3D SR results show that the approach of Pham et al [13] is better than Lanczos interpolation, which is remarkable. Our CNN is even better, surpassing both the Lanczos interpolation and the approach of Pham et al [13].…”

“…We note that our CNN model surpasses Lanczos interpolation in each and every case. Furthermore, our model provides superior results than all the state-of-theart methods [3], [13], [15], [17] considered in our evaluation on the NAMIC data set.…”

“…On the NAMIC data set, we compared our method with the best-performing interpolation method on the CH data set, namely Lanczos interpolation, as well as some stateof-the-art 2D [3], [15], [17] and 3D [13] super-resolution methods. We note that most previous works, including [3], [13], [17], used bicubic interpolation as a relevant baseline. Unlike these works, we opted for Lanczos interpolation, which provided better results than bicubic interpolation and other interpolation methods on the CH data set.…”

“…For our experiments, we used T1-weighted (T1w) and T2-weighted (T2w) images independently. Following [13], we split the NAMIC data set into a training set containing 10 3D images and a test set containing the other 10 images. We kept the same split for T1w and T2w.…”

“…We compared our method with standard resizing methods based on various interpolation schemes, namely nearest neighbors, bilinear, bicubic and Lanczos. In addition to these VOLUME 8, 2020 baselines, we compared with three methods [3], [15], [17] that focused on 2D SISR and one method [13] that focused on 3D SISR. We note that You et al [15] did not report results on NAMIC.…”