Deep learning-Based 3D inpainting of brain MR images

Kang, Seung Kwan; Shin, Seong A.; Seo, Seongho; Byun, Min Soo; Lee, Dong Hoon; Kim, Yu Kyeong; Lee, Jae Sung

doi:10.1038/s41598-020-80930-w

Cited by 26 publications

(18 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To the best of our knowledge, there have been few studies using GAN and CNN motion-correction techniques for VBM [ 17 ]. In particular, no studies have used these techniques for VSRAD analysis.…”

Section: Introductionmentioning

confidence: 99%

Motion correction in MR image for analysis of VSRAD using generative adversarial network

et al. 2022

View full text Add to dashboard Cite

Voxel-based specific region analysis systems for Alzheimer’s disease (VSRAD) are clinically used to measure the atrophied hippocampus captured by magnetic resonance imaging (MRI). However, motion artifacts during acquisition of images may distort the results of the analysis. This study aims to evaluate the usefulness of the Pix2Pix network in motion correction for the input image of VSRAD analysis. Seventy-three patients examined with MRI were distinguished into the training group (n = 51) and the test group (n = 22). To create artifact images, the k-space images were manipulated. Supervised deep learning was employed to obtain a Pix2Pix that generates motion-corrected images, with artifact images as the input data and original images as the reference data. The results of the VSRAD analysis (severity of voxel of interest (VOI) atrophy, the extent of gray matter (GM) atrophy, and extent of VOI atrophy) were recorded for artifact images and motion-corrected images, and were then compared with the original images. For comparison, the image quality of Pix2Pix generated motion-corrected image was also compared with that of U-Net. The Bland-Altman analysis showed that the mean of the limits of agreement was smaller for the motion-corrected images compared to the artifact images, suggesting successful motion correction by the Pix2Pix. The Spearman’s rank correlation coefficients between original and motion-corrected images were almost perfect for all results (severity of VOI atrophy: 0.87–0.99, extent of GM atrophy: 0.88–00.98, extent of VOI atrophy: 0.90–1.00). Pix2Pix generated motion-corrected images that showed generally improved quantitative and qualitative image qualities compared with the U-net generated motion-corrected images. Our findings suggest that motion correction using Pix2Pix is a useful method for VSRAD analysis.

show abstract

Section: Introductionmentioning

confidence: 99%

Motion correction in MR image for analysis of VSRAD using generative adversarial network

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Currently, there is no well-acknowledged method for adding pseudo color to medical images. Previously reported method includes linear color conversion from grayscale to a color map (29), triplicate the grayscale channel to synthesize color image (30,31), concatenating three independent slices from one or cross different series (planes) (32)(33)(34)(35). In this study, we thoroughly benchmarked these methods.…”

Section: Discussionmentioning

confidence: 99%

Classification of Gliomas and Germinomas of the Basal Ganglia by Transfer Learning

Yang

Chen

et al. 2022

Front. Oncol.

View full text Add to dashboard Cite

BackgroundGerm cell tumors (GCTs) are neoplasms derived from reproductive cells, mostly occurring in children and adolescents at 10 to 19 years of age. Intracranial GCTs are classified histologically into germinomas and non-germinomatous germ cell tumors. Germinomas of the basal ganglia are difficult to distinguish based on symptoms or routine MRI images from gliomas, even for experienced neurosurgeons or radiologists. Meanwhile, intracranial germinoma has a lower incidence rate than glioma in children and adults. Therefore, we established a model based on pre-trained ResNet18 with transfer learning to better identify germinomas of the basal ganglia.MethodsThis retrospective study enrolled 73 patients diagnosed with germinoma or glioma of the basal ganglia. Brain lesions were manually segmented based on both T1C and T2 FLAIR sequences. The T1C sequence was used to build the tumor classification model. A 2D convolutional architecture and transfer learning were implemented. ResNet18 from ImageNet was retrained on the MRI images of our cohort. Class activation mapping was applied for the model visualization.ResultsThe model was trained using five-fold cross-validation, achieving a mean AUC of 0.88. By analyzing the class activation map, we found that the model’s attention was focused on the peri-tumoral edema region of gliomas and tumor bulk for germinomas, indicating that differences in these regions may help discriminate these tumors.ConclusionsThis study showed that the T1C-based transfer learning model could accurately distinguish germinomas from gliomas of the basal ganglia preoperatively.

show abstract

“…Even though the use of edges succeeded in improving the performance, it does not provide deeper knowledge of organ structures in the body, resulting in still poor quality of restoration. Recently, a deep neural network for medical inpainting has been proposed in [26]. This framework generates 3D images from sparsely sampled 2D images.…”

Section: Learning-based Approachmentioning

confidence: 99%

“…However, because of ignoring boundary information in training, their method meets the problem of boundary artifacts. Additionally, since [26] was trained and tested on a dataset that is not publicly available, it is hard to compare performance with this study.…”

Section: Learning-based Approachmentioning

confidence: 99%

Multi-Task Learning for Medical Image Inpainting Based on Organ Boundary Awareness

et al. 2021

View full text Add to dashboard Cite

Distorted medical images can significantly hamper medical diagnosis, notably in the analysis of Computer Tomography (CT) images and organ segmentation specifics. Therefore, improving diagnostic imagery accuracy and reconstructing damaged portions are important for medical diagnosis. Recently, these issues have been studied extensively in the field of medical image inpainting. Inpainting techniques are emerging in medical image analysis since local deformations in medical modalities are common because of various factors such as metallic implants, foreign objects or specular reflections during the image captures. The completion of such missing or distorted regions is important for the enhancement of post-processing tasks such as segmentation or classification. In this paper, a novel framework for medical image inpainting is presented by using a multi-task learning model for CT images targeting the learning of the shape and structure of the organs of interest. This novelty has been accomplished through simultaneous training for the prediction of edges and organ boundaries with the image inpainting, while state-of-the-art methods still focus only on the inpainting area without considering the global structure of the target organ. Therefore, our model reproduces medical images with sharp contours and exact organ locations. Consequently, our technique generates more realistic and believable images compared to other approaches. Additionally, in quantitative evaluation, the proposed method achieved the best results in the literature so far, which include a PSNR value of 43.44 dB and SSIM of 0.9818 for the square-shaped regions; a PSNR value of 38.06 dB and SSIM of 0.9746 for the arbitrary-shaped regions. The proposed model generates the sharp and clear images for inpainting by learning the detailed structure of organs. Our method was able to show how promising the method is when applying it in medical image analysis, where the completion of missing or distorted regions is still a challenging task.

show abstract

Deep learning-Based 3D inpainting of brain MR images

Cited by 26 publications

References 54 publications

Motion correction in MR image for analysis of VSRAD using generative adversarial network

Motion correction in MR image for analysis of VSRAD using generative adversarial network

Classification of Gliomas and Germinomas of the Basal Ganglia by Transfer Learning

Multi-Task Learning for Medical Image Inpainting Based on Organ Boundary Awareness

Contact Info

Product

Resources

About