Single Low-Dose CT Image Denoising Using a Generative Adversarial Network With Modified U-Net Generator and Multi-Level Discriminator

Abstract

Low-dose CT (LDCT) images have been widely applied in the medical imaging field due to the potential risk of exposing patients to X-ray radiations. Given the fact that reducing the radiation dose may result in increased noise and artifacts, methods that can eliminate the noise and artifacts in the LDCT image have drawn increasing attentions and produced impressive results over the past decades. However, recent proposed methods mostly suffer from noise remaining, over-smoothing structures or false lesions derived from noise. In this paper, we propose a generative adversarial network (GAN) with novel architecture and loss function for restoring the LDCT image. Firstly, the inception-residual block and residual mapping are incoporated in the U-Net structure. The modified U-Net is applied as the generator of the GAN network so that the noise feature can be eliminated during the forward propagation. Secondly, a novel multi-level joint discriminator is designed by concatenating multiple convolutional neural networks (CNNs) where the output of each deconvolutional layer in the generator is compared with the corresponding down-sampled ground truth image. The adversarial training can be sensitive to noise and artifacts in different scales with this discriminator. Thirdly, we novely define a loss function consisting of the least square adversarial loss, VGG based perceptual loss, MSE based pixel loss and the noise loss, so that the differences in pixel, visual perception and noise distribution are comprehensively considered to optimize the network. Experimental results on both simulated and official simulated clinical images have demonstrated that the proposed method can provide superior performance to the state-of-the-art methods in noise removal, structure preservation and false lesions elimination. INDEX TERMS Low-dose CT image denoising, deep learning, generative adversarial network, inception block, residual mapping, joint loss.

A review on Deep Learning approaches for low-dose Computed Tomography restoration

“…Besides, the data-driven learning method can adapt to any noise type effectively [ 83 ]. Hence, it improves the overall performance of LDCT restoration and possesses a novel advantage over other LDCT restoration methods [ 6 , 46 ].…”

“…Different from residual connections, the concatenation type skip connections in U-net allow transferring of more feature information forward, and it is a significant performance aspect in U-net architecture [ 44 ]. Furthermore, it has been observed that almost all of the U-net-based LDCT restoration applications reviewed in this study have been published by integrating U-net with the Generative Adversarial Network (GAN) s [ 6 , 45 ]. However, after publishing the Pix-to-Pix GAN by Isola et al [ 31 ], there were several LDCT restoration applications published based on it.…”

“…This basic structure of GAN is known as vanilla GAN. Moreover, GAN is flexible to implement different generator models based on various CNN architectures, such as the encoder–decoder [ 58 , 67 ], U-Net [ 6 , 53 , 75 , 79 ], and ResNet [ 12 , 28 , 46 ]. Also, the discriminator mostly acts as a binary classifier to distinguish the synthetic and NDCT images apart.…”

“…Generating some distorted results [ 83 ] SAGAN, Yi, Babyn [ 79 ] Vanilla Gan Patches of size 256 × 256 UNet256 [ 31 ] including 21 convolution layers and 2 deconvolution layers) 9 bypass connections in 9 residual blocks Yes Patch-GAN structure of [ 31 ] (Patch size is 70 × 70) L5, L9 Ensures the sharpness The sensitivity of the sharpness metric is low in some low-contrast regions. False lesion issue [ 6 ] CPCE- 2D/3D, Shan et al [ 58 ] Vanilla Gan Patches of size 64 × 64 3D conveying path-based convolutional encoder-decoder network. 4 encoder and 4 deconvolution layers Three conveying paths between the corresponding encoder and decoder layers No CNN with 6 convolutional layers L5, L8 2D-to-3D transfer learning method is efficient and stable [ 72 ] Loss of texture and tissue details [ 21 ] SMGAN, You et al [ 83 ] W-GAN Voxels of size 80 × 80 × 11 3D CNN with 8 convolution layers No No CNN with 6 convolutional layers L4, L6, L7 It successfully preserves the critical features to be diagnosed Edge blurring and structural dissimilarities appeared in some cases Kang et al [ 35 ] Cycle GAN Patches of size 56 × 56 CNN with 20 convolution layers 6 bypass connections in 6 convolution blocks Yes Patch-GAN structure of [ 31 ] (Patch size is 70 × 70) L5, L10, L11 …”

“…Du et al [ 12 ] were the first team who have applied the attention network to overcome the over-smoothing caused by MSE loss function in current DL-based CT restoration models. The generated attention map of this study was used as prior knowledge about noise distribution over the input image and the implemented visual attention block sustained in the proposed restoration model not only to preserve the fine structures (lesions and other subtle structures) with perceptual similarity but also to explicitly assess the local consistency of the recovered regions [ 6 ].…”