Image super-resolution using progressive generative adversarial networks for medical image analysis

Mahapatra, Dwarikanath; Bozorgtabar, Behzad; Garnavi, Rahil

doi:10.1016/j.compmedimag.2018.10.005

Cited by 241 publications

(73 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To our knowledge, there are only a few previous works [1]- [18] that study the super-resolution of CT or MRI images. Similar to some of these previous works [2]- [7], [9], [11]- [18], we approach single-image super-resolution (SISR) of CT and MRI scans using deep convolutional neural networks (CNNs). We propose a CNN architecture composed of 10 convolutional layers and an intermediate sub-pixel convolutional (upscaling) layer [19] that is placed after the first 6 convolutional layers.…”

Section: Introductionmentioning

confidence: 95%

Convolutional Neural Networks With Intermediate Loss for 3D Super-Resolution of CT and MRI Scans

Ionescu

Verga

2020

IEEE Access

View full text Add to dashboard Cite

Computed Tomography (CT) scanners that are commonly-used in hospitals and medical centers nowadays produce low-resolution images, e.g. one voxel in the image corresponds to at most onecubic millimeter of tissue. In order to accurately segment tumors and make treatment plans, radiologists and oncologists need CT scans of higher resolution. The same problem appears in Magnetic Resonance Imaging (MRI). In this paper, we propose an approach for the single-image super-resolution of 3D CT or MRI scans. Our method is based on deep convolutional neural networks (CNNs) composed of 10 convolutional layers and an intermediate upscaling layer that is placed after the first 6 convolutional layers. Our first CNN, which increases the resolution on two axes (width and height), is followed by a second CNN, which increases the resolution on the third axis (depth). Different from other methods, we compute the loss with respect to the ground-truth high-resolution image right after the upscaling layer, in addition to computing the loss after the last convolutional layer. The intermediate loss forces our network to produce a better output, closer to the ground-truth. A widely-used approach to obtain sharp results is to add Gaussian blur using a fixed standard deviation. In order to avoid overfitting to a fixed standard deviation, we apply Gaussian smoothing with various standard deviations, unlike other approaches. We evaluate the proposed method in the context of 2D and 3D super-resolution of CT and MRI scans from two databases, comparing it to related works from the literature and baselines based on various interpolation schemes, using 2× and 4× scaling factors. The empirical study shows that our approach attains superior results to all other methods. Moreover, our subjective image quality assessment by human observers reveals that both doctors and regular annotators chose our method in favor of Lanczos interpolation in 97.55% cases for an upscaling factor of 2× and in 96.69% cases for an upscaling factor of 4×. In order to allow others to reproduce our state-of-the-art results, we provide our code as open source at https://github.com/lilygeorgescu/3d-super-res-cnn. INDEX TERMSConvolutional neural networks, single-image super-resolution, CT images, MRI images, medical image super-resolution.

show abstract

Section: Introductionmentioning

confidence: 95%

Convolutional Neural Networks With Intermediate Loss for 3D Super-Resolution of CT and MRI Scans

Ionescu

Verga

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…for image-to-image translation of medical images such as the tasks for super resolution [14], noise reduction [15], and cross-modality synthesis [16][17][18]. The idea of pix2pix was proposed by Isola et al [19] based on a conditional GAN that could synthesize images from pairs of precisely aligned image datasets consisting of source and target images-e.g., the CT and structure images fall into the category of source images, while the corresponding RT-dose images are considered target images in the present study.…”

Section: Plos Onementioning

confidence: 99%

Fully automated dose prediction using generative adversarial networks in prostate cancer patients

et al. 2020

View full text Add to dashboard Cite

Although dose prediction for intensity modulated radiation therapy (IMRT) has been accomplished by a deep learning approach, delineation of some structures is needed for the prediction. We sought to develop a fully automated dose-generation framework for IMRT of prostate cancer by entering the patient CT datasets without the contour information into a generative adversarial network (GAN) and to compare its prediction performance to a conventional prediction model trained from patient contours.

show abstract

“…In Ref. [21], a progressive GAN was used to perform the SRR method for medical image analysis. In this study, LR images were converted into HR images using a retinal color fundus images dataset.…”

Section: Related Work a Super-resolution Reconstructionmentioning

confidence: 99%

“…With deeplearning method [17][18][19][20][21][22][23] The method extracts suitable features in various camera settings and environments Large-scale data collection, training, and processing require more time.…”

Section: Visible-light Imagementioning

confidence: 99%

Thermal Image Reconstruction Using Deep Learning

et al. 2020

View full text Add to dashboard Cite

A high-resolution thermal camera is very expensive and is thus difficult to be used. Furthermore, thermal images become blurred in various cases of object motion, camera shaking, and camera defocusing. To solve these problems, a previous super-resolution restoration (SRR) technique converting a thermal image acquired by a low-resolution camera into a highresolution one, and a thermal image deblurring method have been researched. However, existing studies were performed based on 1-channel (grayscale) images. In addition, a large-sized and whole image has been used in the existing thermal image deblurring methods, which causes lower deblurring performance. In this study, we propose novel SRR and deblurring methods. The proposed deblurring method is conducted based on small region images. The proposed methods are also conducted using 3-channel (color) thermal images and generative adversarial networks. In addition, the performances of this method are compared in various color spaces (RGB, Gray, HLS, HSV, Lab, Luv, XYZ, YCrCb), image sizes, and thermal databases. Through experiments using selfcollected databases and open databases, it was confirmed that the proposed methods show better performance than the state-ofthe-art methods. INDEX TERMS thermal image; super-resolution reconstruction; deep learning; generative adversarial network; image deblurring.

show abstract

Image super-resolution using progressive generative adversarial networks for medical image analysis

Cited by 241 publications

References 22 publications

Convolutional Neural Networks With Intermediate Loss for 3D Super-Resolution of CT and MRI Scans

Convolutional Neural Networks With Intermediate Loss for 3D Super-Resolution of CT and MRI Scans

Fully automated dose prediction using generative adversarial networks in prostate cancer patients

Thermal Image Reconstruction Using Deep Learning

Contact Info

Product

Resources

About